From e58192ccefc2706b0af9ba123a8335f54a4085e2 Mon Sep 17 00:00:00 2001 From: Darren Paxton Date: Tue, 22 Sep 2026 10:03:21 -0400 Subject: [PATCH 01/12] feat(runtime): resolve the container CLI once instead of hard-coding docker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every script that ran a container said `docker` outright, so the kit only ran where Docker was installed — even though nothing it does is Docker-specific. scripts/_runtime.sh resolves the CLI once into $RUNTIME and every call site reads that instead. Supported: docker, podman, nerdctl, finch. The contract is narrow and stated at the top of the file — a client CLI speaking Docker's command grammar (run/pull/build/info/version/image inspect) that also ships a `compose` subcommand, since the whole stack is driven through Compose. Anything meeting it can be added to _RUNTIME_SUPPORTED without touching another line. docker is first in the precedence list deliberately: a machine that has always had Docker behaves exactly as it did before this file existed. RUNTIME is read from the environment or .env rather than installed as a `docker` shim on PATH — a shim is per-machine, invisible in the repo, and changes what every other tool on the box does too. This knob is per-checkout and sits next to BUILDER_TAG and DUPLO_TARGET where a reader of these scripts already looks. runc is rejected with an explanation rather than accepted. It is a low-level OCI runtime that executes an already-unpacked bundle by path and cannot resolve an image name, so `runc run busybox:1.36` can never work; it is the layer these CLIs call underneath. Choosing it is a different knob from choosing the client. Two things podman forced that are not cosmetic: docker-compose.yml the builder image default was a nested ${BUILDER_IMAGE:-…:${BUILDER_TAG:-latest}}. podman-compose 1.5 stops at the first '}' and appends the remainder literally, giving an "invalid reference format" even when BUILDER_IMAGE is set. Collapsed to one level. Nothing is lost in any scripted path: builder_resolve_image resolves BUILDER_TAG into BUILDER_IMAGE and exports it before compose is ever invoked, so the default is reached only by a hand-run `compose run`. _builder.sh the builder's uid/gid is no longer always `id -u`. Under rootless podman the caller is already mapped to uid 0 inside the user namespace, so 0:0 is what yields caller-owned files while a real uid maps to an unwritable subuid. runtime_builder_ids picks per runtime. tests/test-runtime.sh covers resolution, rejection, value cleaning and the rootless-podman ids, and sweeps the kit for scripts that still invoke a runtime by name or use it without resolving first. 16 tests. Co-Authored-By: Claude Opus 5 (1M context) --- .env.example | 25 ++- README.md | 5 +- docker-compose.yml | 27 +++- docs/getting-started/prerequisites.md | 38 +++-- docs/troubleshooting.md | 32 +++- logs.sh | 4 +- run.sh | 47 +++--- scripts/_builder.sh | 137 +++++++++------- scripts/_runtime.sh | 219 ++++++++++++++++++++++++++ scripts/detect-bedrock.sh | 20 ++- scripts/switch-llm.sh | 6 +- stop.sh | 6 +- tests/test-runtime.sh | 138 ++++++++++++++++ 13 files changed, 590 insertions(+), 114 deletions(-) create mode 100644 scripts/_runtime.sh create mode 100755 tests/test-runtime.sh diff --git a/.env.example b/.env.example index c8de660..cea4aa0 100644 --- a/.env.example +++ b/.env.example @@ -22,12 +22,33 @@ XTERM_TAG=dev-0.1.2-6cdf181 QDRANT_PORT=6333 QDRANT_API_KEY=devkit-local-qdrant +# ───────────────────────────────────────────────────────────────────────────── +# Container runtime — which CLI runs the stack and the builds. +# +# Leave it commented out to auto-detect, which tries docker, podman, nerdctl, finch in that order (so +# a machine that has always had Docker keeps behaving exactly as before). Set it to pin one: +# +# docker | podman | nerdctl | finch +# +# The requirement is a Docker-compatible CLI that also has a `compose` subcommand — podman delegates +# that to podman-compose or docker-compose, so install one of those alongside it. +# +# NOT a valid value: runc (or crun, youki, runsc…). Those are low-level OCI runtimes that execute an +# already-unpacked bundle given by path — they have no images, registries, networks or compose, so +# there is no `runc run `. They are what the CLIs above call underneath; to choose one, tell +# your CLI (e.g. `podman --runtime crun`), not this variable. +# +# Rootless podman is handled automatically: builds run as uid 0 inside the user namespace, which is +# what makes the extension bundle come out owned by you rather than by an unreachable subuid. +# ───────────────────────────────────────────────────────────────────────────── +#RUNTIME=docker + # Extension build toolchain (.NET SDK 8 + Node 22). Only pulled the first time you build an # extension — the platform itself doesn't use it. An already-pulled tag is reused without checking the # registry: BUILDER_PULL=1 forces a re-pull (this tag moves), BUILDER_IMAGE= replaces the image # outright, BUILDER_PLATFORM= forces an architecture. Each build's package caches live in -# per-uid named volumes that nothing prunes — `docker volume ls | grep duplo_devkit` to find and -# `docker volume rm` to reclaim them (each holds a full NuGet + npm closure). +# per-uid named volumes that nothing prunes — ` volume ls | grep duplo_devkit` to find and +# ` volume rm` to reclaim them (each holds a full NuGet + npm closure). BUILDER_TAG=latest # Build on THIS machine instead of in the toolchain container. Set to 1 if you already have .NET SDK 8 diff --git a/README.md b/README.md index 7591daa..9e9028f 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ [![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE) [![Release](https://img.shields.io/github/v/release/duplocloud/devkit?label=release)](https://github.com/duplocloud/devkit/releases) -Run the DuploCloud AI HelpDesk platform on your laptop via Docker — then build your own +Run the DuploCloud AI HelpDesk platform on your laptop in containers — then build your own **Agent** against it. Each Agent you write ships a backend, a UI, and its own provisioning, and hot-loads into the running platform with no restart. @@ -20,7 +20,8 @@ hot-loads into the running platform with no restart. ## Quick start -You need **Docker with Compose v2**, **Python 3**, and access to an LLM. `./run.sh` prompts for +You need **a container runtime with Compose v2** (docker, podman, nerdctl or finch — set `RUNTIME` +in `.env` to pin one, otherwise it is auto-detected), **Python 3**, and access to an LLM. `./run.sh` prompts for Anthropic, AWS Bedrock, or an Anthropic-compatible LLM gateway (OpenRouter, Bifrost, LiteLLM, …). ```bash diff --git a/docker-compose.yml b/docker-compose.yml index b3e32fd..d3c3c78 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -157,34 +157,45 @@ services: environment: - CORS_ORIGIN=* - # Build toolchain for extensions (.NET SDK 8 + Node 22). Behind a profile so `docker compose pull` + # Build toolchain for extensions (.NET SDK 8 + Node 22). Behind a profile so the `compose pull` # and `up` in run.sh skip it entirely — a user who only runs the platform never downloads it. The - # build wrapper starts it on demand with `docker compose run --rm --no-deps builder`. + # build wrapper starts it on demand with `$RUNTIME compose run --rm --no-deps builder`. # - # Being a compose service (rather than a bare `docker run`) is what lets the build reach the studio over + # Being a compose service (rather than a bare `run`) is what lets the build reach the studio over # the compose network — the SDK feed comes from AIStudio at http://duplo-ai-studio:60021 — but only when - # the studio is in THIS compose project. `docker compose` is project-scoped (the project defaults to the + # the studio is in THIS compose project. Compose is project-scoped (the project defaults to the # directory name), so a platform started from a second checkout or a git worktree — which is how this # repo is developed — is not on this network and that name does not resolve. scripts/_builder.sh detects # which case it is and falls back to the published host port via host.docker.internal (see extra_hosts # below). Both routes are live; the compose network is simply the preferred one. builder: profiles: [tools] - image: ${BUILDER_IMAGE:-quay.io/duplocloud/duplo-extension-builder:${BUILDER_TAG:-latest}} + # Deliberately ONE level of interpolation, not a nested ${BUILDER_IMAGE:-…:${BUILDER_TAG:-latest}}: + # podman-compose 1.5 stops at the first '}' and appends the remainder literally, yielding + # "…busybox:1.36}" and an "invalid reference format" error even when BUILDER_IMAGE is set. Nothing + # is lost — BUILDER_TAG is resolved into BUILDER_IMAGE by scripts/_builder.sh:builder_resolve_image, + # which always exports it before compose is invoked, so the default below is only ever reached by a + # hand-run `compose run` with no BUILDER_IMAGE set (where it now means :latest regardless of + # BUILDER_TAG — use the build scripts, or set BUILDER_IMAGE outright, to pin another tag). + image: ${BUILDER_IMAGE:-quay.io/duplocloud/duplo-extension-builder:latest} # Unset by default, so the build runs on the host's native architecture. Set BUILDER_PLATFORM only # to force one (e.g. linux/amd64 if a publish ever ends up amd64-only) — an unset value is dropped # by compose, whereas copying duplo-ai-studio's linux/amd64 default would silently emulate a .NET # build on Apple Silicon. platform: ${BUILDER_PLATFORM:-} - # The caller's real uid/gid, exported by scripts/_builder.sh — so dist/ ends up owned by the - # invoking user, not root. The default is only a fallback for a hand-run `docker compose run`. + # The ids that make dist/ come out owned by the invoking user, exported by scripts/_builder.sh. + # NOT always `id -u`: under rootless podman the caller is already mapped to uid 0 inside the user + # namespace, so 0:0 is what yields caller-owned files and a real uid maps to an unwritable subuid. + # scripts/_runtime.sh:runtime_builder_ids picks per runtime. The default here is only a fallback for + # a hand-run `compose run`, and is correct for docker; under rootless podman pass DUPLO_UID=0. user: "${DUPLO_UID:-1000}:${DUPLO_GID:-1000}" working_dir: /work # Lets the build reach a studio that is NOT in this compose project via the published host port. # compose scopes services per project (the directory name), so a stack started in another checkout # is not on this project's network and duplo-ai-studio does not resolve; _builder.sh detects that and # points DUPLO_BASE at host.docker.internal instead. A no-op on Docker Desktop, where the name - # already resolves; on Linux it is what makes it resolve at all. + # already resolves; on Linux (and under podman, which maps host-gateway the same way) it is what + # makes it resolve at all. extra_hosts: - "host.docker.internal:host-gateway" volumes: diff --git a/docs/getting-started/prerequisites.md b/docs/getting-started/prerequisites.md index e086dc2..fecffa5 100644 --- a/docs/getting-started/prerequisites.md +++ b/docs/getting-started/prerequisites.md @@ -1,23 +1,24 @@ # 0. Prerequisites -**What you'll do:** Confirm you have what this kit needs — Docker with Compose v2, Python 3, LLM access, -a verified email address, and five free ports. +**What you'll do:** Confirm you have what this kit needs — a container runtime with Compose v2, Python 3, +LLM access, a verified email address, and five free ports. **What you need first:** Nothing. This is the first page. The dev kit runs the whole platform from published images. There is no *language* toolchain to install — no .NET, no Node: every container it starts, and every extension you later build, is built and run inside -Docker. The two things that must exist on the host are Docker itself and `python3`, which the setup -scripts use as their JSON and `.env` editor. +a container. The two things that must exist on the host are a container runtime and `python3`, which the +setup scripts use as their JSON and `.env` editor. --- -## 0.1 Docker, with Compose v2 +## 0.1 A container runtime, with Compose v2 -Docker Desktop, Colima, Rancher Desktop, or a plain Docker Engine all work. What matters is that the -daemon is running and that `docker compose` (two words, the v2 plugin) exists. +Docker is the default and needs no configuration. **podman**, **nerdctl** and **finch** are also +supported — Docker Desktop, Colima, Rancher Desktop, plain Docker Engine and rootless podman all work. +What matters is that the runtime is reachable and that its `compose` subcommand (two words) exists. -1. Check both at once. +1. Check both at once — substitute your runtime for `docker`. ```bash docker --version && docker compose version @@ -33,6 +34,24 @@ daemon is running and that `docker compose` (two words, the v2 plugin) exists. Any Compose `v2.x` or newer is fine. If the second line does not print, you have Compose v1 only and need the v2 plugin. +2. If you have more than one runtime installed, or want to pin one, set `RUNTIME` in `.env`: + + ```bash + RUNTIME=podman # docker | podman | nerdctl | finch + ``` + + Left unset, the scripts auto-detect, trying `docker`, `podman`, `nerdctl`, `finch` in that order. + +> **podman:** install the compose provider too — podman shells out to one rather than implementing +> compose itself (`sudo apt-get install podman-compose`, or `brew install podman-compose`). Rootless +> podman needs nothing else; the build scripts handle the user-namespace uid mapping for you, so +> extension bundles come out owned by you rather than by root. +> +> **Not a runtime:** `runc` (and `crun`, `youki`, `runsc`). Those are low-level OCI runtimes that run an +> already-unpacked bundle by path — they have no images, registries or compose, so they cannot drive +> this kit. They are what the CLIs above use underneath; select one via your CLI, e.g. +> `podman --runtime crun`. + ## 0.2 Python 3 `run.sh` and everything under `scripts/` shell out to `python3` to rewrite `.env` safely (tokens and keys @@ -46,7 +65,8 @@ python3 --version macOS ships one with the Xcode command line tools (`xcode-select --install`) or `brew install python3`; Debian/Ubuntu, `sudo apt-get install -y python3`; RHEL/Amazon Linux, `sudo dnf install -y python3`. -`./run.sh` checks for this and for Docker before it does anything else, and names whatever is missing. +`./run.sh` checks for this and for your container runtime before it does anything else, and names +whatever is missing. ## 0.3 LLM access diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 92b89cd..5662a3c 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -32,8 +32,10 @@ This kit needs a couple of things that aren't here yet: ``` **Why** — `run.sh` checks its two host prerequisites before it touches anything: `python3` (it is the -`.env` editor and the JSON parser for every API call the setup makes) and `docker`, including the -`docker compose` v2 *subcommand* — the standalone `docker-compose` v1 binary does not satisfy it. Every +`.env` editor and the JSON parser for every API call the setup makes) and a container runtime +(`docker`, `podman`, `nerdctl` or `finch` — auto-detected, or pinned with `RUNTIME` in `.env`), +including its `compose` v2 *subcommand* — the standalone `docker-compose` v1 binary does not satisfy +it, and podman needs a compose provider installed alongside it (`podman-compose`). Every missing item is listed in one pass, so the list is the whole list. **Fix** — install what it names, per [prerequisites](getting-started/prerequisites.md), and re-run. @@ -45,8 +47,8 @@ missing item is listed in one pass, so the list is the whole list. python3 --version && docker compose version ``` -A *stopped* Docker daemon is only a warning at this stage; it becomes a real error later, at -`docker compose pull`. +An unreachable runtime is only a warning at this stage; it becomes a real error later, at +`compose pull`. ### A port is already in use @@ -62,7 +64,7 @@ but a port can still be taken by something else on your machine. The defaults ar `UI_PORT=4210`, `AGENT_PORT=8010`, `MONGO_PORT=27018`, `XTERM_PORT=6061`, `QDRANT_PORT=6333`. `QDRANT_PORT` is the exception to the offset: 6333 is Qdrant's own default, so a Qdrant you already run -locally will hold it. `docker compose up -d` is not guarded against this — one clashing port aborts the +locally will hold it. `compose up -d` is not guarded against this — one clashing port aborts the whole run — so move it in `.env` before starting. **Fix** — find what holds the port, then either stop it or move the dev kit: @@ -82,10 +84,26 @@ docker compose ps # every service Up curl -fsS http://localhost:4210 >/dev/null && echo "UI is answering" ``` +### podman-specific + +All of these are handled by the scripts; they are listed because you will meet them the moment you run +a `compose` command *by hand* under podman, and the errors do not say what is actually wrong. + +| Symptom | Cause | What to do | +| --- | --- | --- | +| `missing services [builder]` | podman-compose does not implicitly enable a service's `profiles` the way `docker compose run` does, and `builder` sits behind the `tools` profile. `COMPOSE_PROFILES=tools` does **not** help — podman-compose ignores that variable. | Name it: `podman compose --profile tools run --rm --no-deps builder …` | +| `invalid reference format` on an image ending in `}` | podman-compose 1.5 stops at the first `}` of a nested `${A:-${B}}` and appends the rest literally. | Already removed from `docker-compose.yml`; don't reintroduce nesting — `tests/test-runtime.sh` checks for it. | +| `compose ps` exits 2 with a usage dump | podman-compose's `ps` has no `--services` or `--status`; its whole grammar is `-q` and `--format`. | The scripts use `runtime_compose_running_services` (in `scripts/_runtime.sh`), which falls back to filtering on the `com.docker.compose.*` labels that both implementations set. | +| An extension bundle in `dist/` is owned by uid 101000 and you cannot delete it | Rootless podman maps *your* uid to container uid 0, so passing a real `--user 1000` lands on an unwritable subuid. | The build scripts pass `0:0` under rootless podman automatically. For an already-broken directory: `podman unshare rm -rf dist`. | +| `rootless netns: kill network process: permission denied` on container teardown | A podman rootless cleanup quirk. Cosmetic — it is printed after the container has already exited, and the exit status is still propagated correctly. | Ignore it. | + +`host.docker.internal` works under podman: the `extra_hosts: host-gateway` entry in `docker-compose.yml` +maps it, and podman additionally provides `host.containers.internal` for the same address. + ### More entries belong here -Image pull failures and `docker login quay.io`, and insufficient memory or disk. Add them as they are hit — -with the verbatim error string. +Image pull failures and ` login quay.io`, and insufficient memory or disk. Add them as they are +hit — with the verbatim error string. --- diff --git a/logs.sh b/logs.sh index 5394c5a..bf9140f 100755 --- a/logs.sh +++ b/logs.sh @@ -2,4 +2,6 @@ # Tail logs. Optional service name: duplo-ai-studio | claude-code-agent | duplo-ui | mongo | xterm set -euo pipefail cd "$(dirname "$0")" -exec docker compose logs -f "$@" +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman | nerdctl | finch) +runtime_resolve || exit 1 +exec "$RUNTIME" compose logs -f "$@" diff --git a/run.sh b/run.sh index 410fab3..beca595 100755 --- a/run.sh +++ b/run.sh @@ -50,6 +50,7 @@ set -euo pipefail cd "$(dirname "$0")" ENV=.env +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman | nerdctl | finch); resolved in preflight . ./scripts/_provider_gateway.sh . ./scripts/_provider_subscription.sh . ./scripts/_studio_api.sh @@ -109,30 +110,40 @@ done # python3 every .env write and every JSON response in this script and in scripts/ is parsed with it. # No version floor: only the stdlib (json, sys, os, datetime, base64) is used, so anything # still called python3 is new enough. -# docker the whole stack, and every extension build, runs in containers. Compose must be v2, i.e. -# the `docker compose` subcommand — the standalone `docker-compose` v1 binary is not used. +# RUNTIME the whole stack, and every extension build, runs in containers. Which CLI drives them is +# resolved by scripts/_runtime.sh (RUNTIME in the environment or .env, else auto-detected); +# it must provide a v2-style `compose` subcommand. The standalone `docker-compose` v1 binary +# is not used. # -# The daemon-not-running case is a warning, not an error: `docker compose pull` further down reports it -# far better than we can, and failing here would block the flag-only paths that never touch the daemon. +# The daemon-not-running case is a warning, not an error: `$RUNTIME compose pull` further down reports +# it far better than we can, and failing here would block the flag-only paths that never touch it. MISSING="" command -v python3 >/dev/null 2>&1 || MISSING="${MISSING} • python3 — not on PATH. macOS: brew install python3 (or install Xcode command line tools). Debian/Ubuntu: sudo apt-get install -y python3. RHEL/Amazon Linux: sudo dnf install -y python3." -if ! command -v docker >/dev/null 2>&1; then - MISSING="${MISSING} - • docker — not on PATH. Install Docker Desktop (https://docs.docker.com/get-docker/), Colima, or - Rancher Desktop, then re-run." -elif ! docker compose version >/dev/null 2>&1; then - # `docker` exists but has no `compose` subcommand: either Compose v1 only, or a plugin-less install. + +# runtime_resolve must run in THIS shell, not a command substitution: it sets RUNTIME as a side effect, +# which a subshell would discard. So its diagnostics go straight to stderr rather than into $MISSING — +# the one-pass property is preserved by deferring the exit until python3 has been reported too. +RUNTIME_OK=1 +runtime_resolve || RUNTIME_OK=0 +if [ "$RUNTIME_OK" = 1 ] && ! "$RUNTIME" compose version >/dev/null 2>&1; then + # The CLI exists but has no `compose` subcommand: Compose v1 only, or a plugin-less install. For + # podman that usually means the podman-compose (or docker-compose) provider is missing, since podman + # shells out to one rather than implementing compose itself. MISSING="${MISSING} - • docker compose (v2) — 'docker compose version' failed. The standalone docker-compose v1 binary is - not enough; install the Compose v2 plugin, or upgrade Docker Desktop." + • $RUNTIME compose — '$RUNTIME compose version' failed. A v2-style compose subcommand is required; + the standalone docker-compose v1 binary is not enough. + docker — install the Compose v2 plugin, or upgrade Docker Desktop. + podman — install the provider it delegates to: apt install podman-compose (or dnf/brew). + nerdctl/finch — upgrade to a build that ships 'compose'." fi if [ -n "$MISSING" ]; then echo "This kit needs a couple of things that aren't here yet:$MISSING" >&2 - exit 1 + RUNTIME_OK=0 fi -docker info >/dev/null 2>&1 || echo "Note: the Docker daemon doesn't look like it's running — start it before this gets to 'Pulling images'." >&2 +[ "$RUNTIME_OK" = 1 ] || exit 1 +"$RUNTIME" info >/dev/null 2>&1 || echo "Note: $RUNTIME doesn't look ready (daemon not running, or no connection) — sort that out before this gets to 'Pulling images'." >&2 # ── .env helpers (line-based; safe for tokens/keys with special chars) ──────── [ -f "$ENV" ] || { [ -f .env.example ] && cp .env.example "$ENV" || touch "$ENV"; } @@ -582,7 +593,7 @@ fi # whenever that fetch failed. --reset-license (below) is how you ask for it to actually go away. if [ "$RESET" = 1 ]; then echo "==> --reset: tearing down stack + volumes (DB, extensions, file store)…" - docker compose down -v 2>/dev/null || true + "$RUNTIME" compose down -v 2>/dev/null || true for k in Authentication__LocalAdminEmail Authentication__LocalAdminPassword Authentication__SuperUsers \ DEVKIT_MODEL ANTHROPIC_API_KEY AWS_REGION AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN \ CLAUDE_MODEL CLAUDE_EXTRA_MODELS "${GATEWAY_KEYS[@]}" "${SUBSCRIPTION_KEYS[@]}" \ @@ -868,7 +879,7 @@ if [ -z "$MODEL" ]; then fi # CONTAINER_IMDS is ok | blocked | unknown:. Only a definite `blocked` withholds option 3 — # that one means the container test ran and failed (the hop limit), so the role would break at - # runtime. `unknown:` means we couldn't run the test at all (no docker yet, daemon down, busybox + # runtime. `unknown:` means we couldn't run the test at all (no runtime yet, daemon down, busybox # not pullable); the role itself is proven, so offer it with the caveat rather than hiding a # working option because our own check couldn't execute. if [ "$BEDROCK_AVAILABLE" = 1 ] && [ "$CONTAINER_IMDS" != blocked ]; then @@ -997,8 +1008,8 @@ for v in STUDIO_TAG UI_TAG; do done # ── start the stack ─────────────────────────────────────────────────────────── -echo "==> Pulling images…"; docker compose pull -echo "==> Starting…"; docker compose up -d +echo "==> Pulling images…"; "$RUNTIME" compose pull +echo "==> Starting…"; "$RUNTIME" compose up -d STUDIO_PORT="$(getenv STUDIO_PORT)"; [ -z "$STUDIO_PORT" ] && STUDIO_PORT=60021 UI_PORT="$(getenv UI_PORT)"; [ -z "$UI_PORT" ] && UI_PORT=4200 diff --git a/scripts/_builder.sh b/scripts/_builder.sh index 06efb4d..718ae4d 100644 --- a/scripts/_builder.sh +++ b/scripts/_builder.sh @@ -4,13 +4,21 @@ # # Why a container at all: the dev-kit's runtime is entirely containerized, but the build was not — it # hard-required `jq dotnet npm zip curl unzip` on the developer's laptop (and `jq` isn't shipped by -# macOS, so it was an undeclared prerequisite that failed at line 15). Docker Compose is now the only -# prerequisite. +# macOS, so it was an undeclared prerequisite that failed at line 15). A compose-capable container CLI +# is now the only prerequisite. +# +# Every container command here goes through $RUNTIME (see scripts/_runtime.sh, sourced below) rather +# than a literal `docker`, so the same build works under docker, podman, nerdctl or finch. # # REQUIRES scripts/_target.sh, sourced FIRST: this file uses its `_envv` (to read .env for BUILDER_IMAGE / # BUILDER_PULL / BUILDER_TAG / STUDIO_PORT) and its `_TARGET`. Both current callers do source it first; # that is a requirement of sourcing this file, not an accident of their layout. +# BASH_SOURCE, not $0: $0 is whichever script sourced this one, and build-extension.sh may be invoked +# from any cwd. _runtime.sh is idempotent, so sourcing it here as well as in run.sh costs nothing. +# shellcheck source=scripts/_runtime.sh +. "${BASH_SOURCE[0]%/*}/_runtime.sh" + # builder_truthy -> echoes 1 or 0; returns 1 (no output) if isn't a boolean at all. # # Accepts the spellings a human actually types into an env var, case-insensitively, because @@ -26,26 +34,26 @@ builder_truthy() { esac } -# builder_mode -> native|container|error: +# builder_mode -> native|container|error: # # Kept as a pure function of four booleans (no `command -v`, no filesystem probes) so the whole -# decision matrix can be exercised without Docker, a platform, or a toolchain. +# decision matrix can be exercised without a container runtime, a platform, or a toolchain. builder_mode() { # Normalize every argument up front, before any branch is taken, so a bad value can't be silently # read as false by whichever comparison happens to see it first. Separate declaration from assignment: # `local x=$(cmd)` masks the command's exit status behind `local`'s own, so the || would never fire. - local force_native in_container have_toolchain have_docker + local force_native in_container have_toolchain have_runtime force_native=$(builder_truthy "${1-}") || { echo error:badarg; return; } in_container=$(builder_truthy "${2-}") || { echo error:badarg; return; } have_toolchain=$(builder_truthy "${3-}") || { echo error:badarg; return; } - have_docker=$(builder_truthy "${4-}") || { echo error:badarg; return; } + have_runtime=$(builder_truthy "${4-}") || { echo error:badarg; return; } # The explicit escape hatch outranks everything: --native / DUPLO_BUILD_NATIVE=1 means the caller # has their own toolchain and wants it used, so don't second-guess them. [ "$force_native" = 1 ] && { echo native; return; } - # Already inside a container. Nesting is not an option — the agent container has the docker CLI but NO - # /var/run/docker.sock, so builder_probe_docker's daemon check fails — and it isn't needed either, + # Already inside a container. Nesting is not an option — the agent container has a container CLI but + # no socket to talk to, so builder_probe_runtime's reachability check fails — and it isn't needed either, # because the agent image ships the whole toolchain. This is the branch the agent's build takes. It is # also the branch the re-launched build takes inside the builder image, which stops the wrapper recursing. if [ "$in_container" = 1 ]; then @@ -53,14 +61,14 @@ builder_mode() { echo error:toolchain; return fi - # A host with Docker: containerize, even if a local toolchain exists, so that laptop, CI and agent - # builds all use one pinned toolchain. `--native` is there for anyone who wants their own. - [ "$have_docker" = 1 ] && { echo container; return; } + # A host with a working runtime: containerize, even if a local toolchain exists, so that laptop, CI + # and agent builds all use one pinned toolchain. `--native` is there for anyone who wants their own. + [ "$have_runtime" = 1 ] && { echo container; return; } - # No Docker. A complete local toolchain still works, so degrade gracefully rather than fail (this - # keeps a Docker-less CI runner building). + # No runtime. A complete local toolchain still works, so degrade gracefully rather than fail (this + # keeps a runtime-less CI runner building). [ "$have_toolchain" = 1 ] && { echo native; return; } - echo error:docker + echo error:runtime } # The tools build-extension.sh has always required. One list, used both to decide (builder_probe_toolchain) @@ -96,21 +104,20 @@ builder_probe_toolchain() { echo 1 } -builder_probe_docker() { - # The CLI existing is not enough — the agent image HAS the docker binary and no daemon, so - # `command -v docker` alone would answer 1 there and send the build looking for a socket that isn't - # mounted. Only a reachable daemon counts. +builder_probe_runtime() { + # The CLI existing is not enough — the agent image HAS a client binary and nothing to talk to, so + # `command -v` alone would answer 1 there and send the build looking for a socket that isn't mounted. + # Only a runtime that actually answers counts. # # `version --format {{.Server.Version}}` rather than `info`: printing the SERVER version requires # contacting the server, so it proves exactly as much, and it is ~25x cheaper (measured 0.042s vs # 1.074s). builder_dispatch evaluates all four probes as *arguments* to builder_mode, so bash runs # this one even when --native has already decided the answer — and build-all.sh --native pays it once - # per extension. - if command -v docker >/dev/null 2>&1 && docker version --format '{{.Server.Version}}' >/dev/null 2>&1; then - echo 1 - else - echo 0 - fi + # per extension. Podman is daemonless, but it implements the same query (reporting its own version as + # the server), so the one probe covers every supported runtime. + [ -n "${RUNTIME:-}" ] || { echo 0; return; } + command -v "$RUNTIME" >/dev/null 2>&1 || { echo 0; return; } + if "$RUNTIME" version --format '{{.Server.Version}}' >/dev/null 2>&1; then echo 1; else echo 0; fi } builder_missing_tools() { @@ -124,8 +131,8 @@ builder_missing_tools() { # --- Image resolution ---------------------------------------------------------------------------- # _envv is a raw `grep | cut`, so whatever follows the `=` in .env survives verbatim: a CRLF-edited file -# yields "v1", a quoted value yields "\"v1\"", trailing spaces stay spaces. Docker rejects all three -# with "invalid reference format". Strip the three that are unambiguously accidental rather than fail on +# yields "v1", a quoted value yields "\"v1\"", trailing spaces stay spaces. Every runtime rejects all +# three with "invalid reference format". Strip the three that are unambiguously accidental rather than fail on # them — nobody means to pin a tag whose name contains a carriage return. builder_clean_value() { local v="${1-}" @@ -137,7 +144,7 @@ builder_clean_value() { printf '%s' "$v" } -# Resolve BUILDER_IMAGE, pulling (or, if that fails, building locally) so `docker compose run` finds it. +# Resolve BUILDER_IMAGE, pulling (or, if that fails, building locally) so `$RUNTIME compose run` finds it. # A local fallback keeps the dev-kit usable before the image is published, and for anyone without quay # access. Every success path EXPORTS the name: compose reads it from the environment, and an unexported # one means compose silently falls back to the `${BUILDER_IMAGE:-quay…}` default in docker-compose.yml — @@ -146,7 +153,7 @@ builder_clean_value() { # Caching: an image that is already local is used as-is, without contacting the registry, so a warm build # never pays for a pull. The cost is that a MOVING tag is never refreshed — and BUILDER_TAG=latest, our # default, is exactly that, on an image whose content is a .NET SDK. BUILDER_PULL=1 forces the re-pull; -# without it the only recourse would be `docker rmi`. +# without it the only recourse would be `$RUNTIME rmi`. # # Both knobs are read from the environment AND from .env (via _envv), because compose interpolates # docker-compose.yml against .env too: if .env's BUILDER_IMAGE were ignored here, compose would honour it @@ -155,9 +162,9 @@ builder_clean_value() { # error — "offline with BUILDER_PULL=1 set" is the ordinary way to get here, and a stale toolchain beats no # build. Returns 0 when BUILDER_IMAGE is usable as-is. builder_keep_local_copy() { # builder_keep_local_copy - docker image inspect "$BUILDER_IMAGE" >/dev/null 2>&1 || return 1 + "$RUNTIME" image inspect "$BUILDER_IMAGE" >/dev/null 2>&1 || return 1 echo "WARNING: could not re-pull $BUILDER_IMAGE, using the local copy." >&2 - echo " docker pull said: $(printf '%s' "${1-}" | tail -1)" >&2 + echo " $RUNTIME pull said: $(printf '%s' "${1-}" | tail -1)" >&2 } builder_resolve_image() { @@ -174,18 +181,18 @@ builder_resolve_image() { img="$(builder_clean_value "${BUILDER_IMAGE:-$(_envv BUILDER_IMAGE)}")" if [ -n "$img" ]; then BUILDER_IMAGE="$img"; export BUILDER_IMAGE - if [ "$force_pull" != 1 ] && docker image inspect "$BUILDER_IMAGE" >/dev/null 2>&1; then return 0; fi - pull_err="$(docker pull -q "$BUILDER_IMAGE" 2>&1)" && return 0 + if [ "$force_pull" != 1 ] && "$RUNTIME" image inspect "$BUILDER_IMAGE" >/dev/null 2>&1; then return 0; fi + pull_err="$("$RUNTIME" pull -q "$BUILDER_IMAGE" 2>&1)" && return 0 builder_keep_local_copy "$pull_err" && return 0 # A pinned image that cannot be obtained is fatal: the caller named one specific image, so quietly # building a different one would not be doing what they asked. echo "ERROR: BUILDER_IMAGE=$BUILDER_IMAGE is neither present locally nor pullable." >&2 - echo " docker pull said: $(printf '%s' "$pull_err" | tail -1)" >&2 + echo " $RUNTIME pull said: $(printf '%s' "$pull_err" | tail -1)" >&2 return 1 fi local tag; tag="$(builder_clean_value "$(_envv BUILDER_TAG)")"; tag="${tag:-latest}" - # Docker's own tag grammar: [A-Za-z0-9_][A-Za-z0-9._-]{0,127}. Checked here because the alternative is + # OCI tag grammar: [A-Za-z0-9_][A-Za-z0-9._-]{0,127}. Checked here because the alternative is # a pull that fails for an unobvious reason and a silent fall-through to building something else. case "$tag" in *[!A-Za-z0-9._-]* | [!A-Za-z0-9_]* ) @@ -195,24 +202,24 @@ builder_resolve_image() { return 1 ;; esac BUILDER_IMAGE="quay.io/duplocloud/duplo-extension-builder:$tag"; export BUILDER_IMAGE - if [ "$force_pull" != 1 ] && docker image inspect "$BUILDER_IMAGE" >/dev/null 2>&1; then return 0; fi + if [ "$force_pull" != 1 ] && "$RUNTIME" image inspect "$BUILDER_IMAGE" >/dev/null 2>&1; then return 0; fi echo "==> Fetching the build toolchain image ($BUILDER_IMAGE)" >&2 - pull_err="$(docker pull -q "$BUILDER_IMAGE" 2>&1)" && return 0 + pull_err="$("$RUNTIME" pull -q "$BUILDER_IMAGE" 2>&1)" && return 0 builder_keep_local_copy "$pull_err" && return 0 # Not published, or no quay access — build it from the Dockerfile that ships in this repo. `build` is # the context, not `.`: the Dockerfile COPYs nothing from the context, so sending the whole repo - # (samples/, packages/, node_modules/) to the daemon would be pure waste. + # (samples/, packages/, node_modules/) to the builder would be pure waste. # # The pull error is reported, not swallowed: "unauthorized", "manifest unknown" and a registry rate # limit all land here, and they are the difference between "the image isn't published yet" (fine, build # it) and "your credentials expired" (which a two-minute local build silently papers over). echo " pull failed: $(printf '%s' "$pull_err" | tail -1)" >&2 BUILDER_IMAGE="duplo-extension-builder:local"; export BUILDER_IMAGE - docker image inspect "$BUILDER_IMAGE" >/dev/null 2>&1 && return 0 + "$RUNTIME" image inspect "$BUILDER_IMAGE" >/dev/null 2>&1 && return 0 echo " building it locally from build/Dockerfile.builder instead (one time, ~2 min)" >&2 - docker build -q -f build/Dockerfile.builder -t "$BUILDER_IMAGE" build >/dev/null \ + "$RUNTIME" build -q -f build/Dockerfile.builder -t "$BUILDER_IMAGE" build >/dev/null \ || { echo "ERROR: could not build the builder image from build/Dockerfile.builder" >&2; return 1; } } @@ -220,17 +227,18 @@ builder_resolve_image() { # builder_studio_state -> in-project | absent | unknown # -# One `docker compose ps` (~0.06s isolated, ~0.11s amortized against a build) — this sits in front of -# every containerized build, so it has to stay a single call. +# One runtime_compose_running_services call (~0.06s isolated, ~0.11s amortized against a build) — this +# sits in front of every containerized build, so it has to stay a single call. That helper is also what +# keeps the answer correct on podman, whose `compose ps` has no --status/--services flags. # -# `docker compose` is PROJECT-scoped, and the project defaults to the directory name. So this answers a +# Compose is PROJECT-scoped, and the project defaults to the directory name. So this answers a # narrower question than "is the platform up": it answers "is the studio on the network this build will # join". A second checkout (or a git worktree, which is how this repo is developed) is a different # project, and its running studio is invisible here — hence three states, not two: `absent` is evidence, # `unknown` (the query itself failed) is not, and they must not lead to the same message. builder_studio_state() { local running - running="$(docker compose ps --status running --services 2>/dev/null)" || { echo unknown; return; } + running="$(runtime_compose_running_services)" || { echo unknown; return; } if printf '%s\n' "$running" | grep -qx duplo-ai-studio; then echo in-project; else echo absent; fi } @@ -257,7 +265,7 @@ builder_studio_url() { # success rather than a guess, so this says what it is doing instead of refusing to try. `unknown` # stays silent: a scary message in front of every build, on no evidence, teaches people to ignore it. if [ "$state" = absent ]; then - echo "WARNING: no running duplo-ai-studio in this compose project (docker compose scopes services" >&2 + echo "WARNING: no running duplo-ai-studio in this compose project ($RUNTIME compose scopes services" >&2 echo " per project, and this one is the directory '$(basename "$PWD")'), so the build will" >&2 echo " try the published host port instead: $url" >&2 echo " That reaches a platform started from another checkout, or by other means. If nothing" >&2 @@ -287,25 +295,35 @@ builder_dispatch() { # value we actually resolved, which is not necessarily the one in the environment. local want_native; want_native="$(builder_clean_value "${DUPLO_BUILD_NATIVE:-$(_envv DUPLO_BUILD_NATIVE)}")" + # Resolve the CLI before probing. Failure is only fatal when a runtime was NAMED: the caller said + # which one to use, and quietly building with a different one — or dropping to a native build they + # did not ask for — is not what they asked for. Auto-detect finding nothing stays silent, because + # builder_mode's native fallback below still builds on a machine with the full toolchain, and that is + # what keeps a runtime-less CI runner working. The re-run is to print the specific reason, which was + # suppressed on the first call. + runtime_resolve 2>/dev/null || { + if [ -n "$(runtime_requested)" ]; then runtime_resolve; exit 1; fi + } + local mode; mode="$(builder_mode \ "$want_native" \ "$(builder_probe_in_container)" \ "$(builder_probe_toolchain)" \ - "$(builder_probe_docker)")" + "$(builder_probe_runtime)")" case "$mode" in native) return 0 ;; error:toolchain) echo "ERROR: missing build tool(s): $(builder_missing_tools)" >&2 - echo " This is running inside a container with no Docker daemon, so the build cannot be" >&2 + echo " This is running inside a container with no reachable runtime, so the build cannot be" >&2 echo " delegated. Use an image that ships the toolchain (.NET SDK 8 + Node 22), e.g." >&2 echo " build/Dockerfile.builder in this repo." >&2 exit 1 ;; - error:docker) - echo "ERROR: Docker is not available and the local toolchain is incomplete." >&2 + error:runtime) + echo "ERROR: no container runtime is available and the local toolchain is incomplete." >&2 echo " Missing: $(builder_missing_tools)" >&2 - echo " Install Docker (Compose v2) — that is the only prerequisite — or install the" >&2 - echo " toolchain yourself and re-run with --native." >&2 + echo " Install docker, podman, nerdctl or finch (with a 'compose' subcommand) — that is the" >&2 + echo " only prerequisite — or install the toolchain yourself and re-run with --native." >&2 exit 1 ;; container) ;; # error:badarg, or a mode this function hasn't been taught. The only way to reach it in practice is @@ -348,13 +366,22 @@ builder_dispatch() { builder_resolve_image || exit 1 - # Run as the caller so dist/ is theirs, not root's — BOTH halves: uid alone leaves the group as the - # compose default, i.e. a bundle the caller's group cannot write. - DUPLO_UID="$(id -u)" DUPLO_GID="$(id -g)" + # Run as whichever ids make dist/ come out owned by the caller rather than by root or by an + # unreachable subuid — BOTH halves: uid alone leaves the group as the compose default, i.e. a bundle + # the caller's group cannot write. Which ids those are differs per runtime, so the choice lives in + # runtime_builder_ids (see the long note there: rootless podman needs 0:0, docker needs the real uid). + read -r DUPLO_UID DUPLO_GID <<<"$(runtime_builder_ids)" export DUPLO_UID DUPLO_GID BUILDER_IMAGE DUPLO_BASE DUPLO_TARGET="${_TARGET:-local}" - echo "==> Building in $BUILDER_IMAGE (uid $DUPLO_UID:$DUPLO_GID) — --native to use your own toolchain" + echo "==> Building in $BUILDER_IMAGE via $(runtime_label) (uid $DUPLO_UID:$DUPLO_GID)" \ + "— --native to use your own toolchain" # --rm: the container is a one-shot. --no-deps: never start the platform as a side effect of a build. # -T: nothing in a build wants a TTY, and this is exec'd from a script whose stdio may already be a pipe. - exec docker compose run --rm -T --no-deps builder "$script" "$@" + # + # --profile tools is REQUIRED, not belt-and-braces: the builder service sits behind that profile, and + # while `docker compose run` implicitly enables the profiles of the service it was asked to run, + # podman-compose 1.5 does not — it reports "missing services [builder]" and exits 1. Naming the + # profile explicitly is a no-op on docker and the only way in on podman. (COMPOSE_PROFILES=tools is + # NOT an alternative here: podman-compose ignores that variable.) + exec "$RUNTIME" compose --profile tools run --rm -T --no-deps builder "$script" "$@" } diff --git a/scripts/_runtime.sh b/scripts/_runtime.sh new file mode 100644 index 0000000..050ee41 --- /dev/null +++ b/scripts/_runtime.sh @@ -0,0 +1,219 @@ +# shellcheck shell=bash +# Sourced FIRST by run.sh, stop.sh, logs.sh and scripts/* — resolves the container CLI once, so every +# call site reads `$RUNTIME run` / `$RUNTIME compose` instead of hard-coding `docker`. +# +# Why a variable rather than a `docker` shim on PATH: a shim is per-machine, invisible in the repo, and +# silently changes what every other tool on the box does too. RUNTIME is per-checkout, lives in .env +# next to every other knob (BUILDER_TAG, DUPLO_TARGET…), and is visible in the one place a reader of +# these scripts already looks. +# +# WHAT COUNTS AS A RUNTIME HERE +# Every value below is a *client CLI* that speaks Docker's command grammar — `run`, `pull`, `build`, +# `info`, `version`, `image inspect` — AND ships a `compose` subcommand, because this kit drives the +# whole stack through Compose. That is the entire contract; anything meeting it can be added to +# _RUNTIME_SUPPORTED without touching another line. +# +# docker the default, and what the kit was written against. +# podman daemonless; rootless by default. See the uid note on runtime_builder_ids below. +# nerdctl containerd's CLI. +# finch AWS's macOS/Windows wrapper around nerdctl in a VM. +# +# runc is deliberately NOT here, though it was asked for. It is a *low-level OCI runtime*: it executes +# an already-unpacked bundle (a rootfs plus config.json) given by path, and has no concept of an image, +# a registry, a network or a compose file — `runc run busybox:1.36` cannot work, because runc never +# resolves an image name. It is the layer the CLIs above call *underneath* (this machine's podman drives +# `crun`, runc's C rewrite, via `--runtime`). Choosing the low-level runtime is a different knob from +# choosing the client, so runtime_resolve rejects it with that explanation rather than emitting commands +# that would fail later with something unrecognisable. + +# The order here is also the auto-detect precedence: docker first, so a machine that has always had +# Docker keeps behaving exactly as it did before this file existed. +_RUNTIME_SUPPORTED=(docker podman nerdctl finch) + +# Self-contained .env read. _target.sh defines an identical `_envv`, but this file is sourced by run.sh +# and stop.sh too, which never source _target.sh — and it must work before either has run. Same +# DUPLO_ENV_FILE convention, same `|| true` so a missing key yields empty instead of tripping the +# callers' `set -euo pipefail`. +_RUNTIME_ENV_FILE="${DUPLO_ENV_FILE:-.env}" +_runtime_envv() { grep -E "^$1=" "$_RUNTIME_ENV_FILE" 2>/dev/null | head -1 | cut -d= -f2- || true; } + +# Strip the three accidents a hand- or Windows-edited .env leaves behind. Same three cases, and the same +# reasoning, as builder_clean_value in _builder.sh — duplicated rather than shared because that file is +# sourced only by the two build scripts, and this one has to stand alone. +_runtime_clean() { + local v="${1-}" + v="${v//$'\r'/}" # CRLF line endings + case "$v" in \"*\") v="${v#\"}"; v="${v%\"}" ;; esac # "quoted" + case "$v" in \'*\') v="${v#\'}"; v="${v%\'}" ;; esac # 'quoted' + v="${v#"${v%%[![:space:]]*}"}" # leading whitespace + v="${v%"${v##*[![:space:]]}"}" # trailing whitespace + printf '%s' "$v" +} + +# runtime_detect -> first supported CLI on PATH, or empty. +# +# Presence only — no daemon check. Whether the thing actually answers is a separate question with a +# separate message (run.sh's preflight, builder_probe_runtime), and conflating them here would make an +# unstarted Docker Desktop silently fall through to podman. +runtime_detect() { + local r + for r in "${_RUNTIME_SUPPORTED[@]}"; do + command -v "$r" >/dev/null 2>&1 && { printf '%s' "$r"; return; } + done + printf '' +} + +# runtime_requested -> the explicitly requested runtime (environment, else .env), or empty when the +# choice is being left to auto-detection. +# +# The distinction matters to the build path: "you asked for X and X is unusable" is a hard error, +# because silently building with something else is not what was asked for — whereas "nothing found" +# can legitimately fall back to a native toolchain build. +runtime_requested() { _runtime_clean "${RUNTIME:-$(_runtime_envv RUNTIME)}"; } + +# runtime_resolve -> sets and exports RUNTIME; returns 1 with a message on stderr if it cannot. +# +# Precedence: environment → .env → auto-detect. Exported because the build path hands the choice down +# to a nested `$RUNTIME compose run` invocation of these same scripts, and an unexported RUNTIME there +# would silently re-detect and could pick a different CLI than the one the caller asked for. +# +# Idempotent: sourcing this file twice (build-extension.sh sources _builder.sh, which sources this) must +# not re-run detection or re-print anything. +runtime_resolve() { + [ -n "${_RUNTIME_RESOLVED:-}" ] && return 0 + + local want; want="$(_runtime_clean "${RUNTIME:-$(_runtime_envv RUNTIME)}")" + + if [ -z "$want" ]; then + want="$(runtime_detect)" + if [ -z "$want" ]; then + echo "ERROR: no container runtime found on PATH." >&2 + echo " This kit needs one of: ${_RUNTIME_SUPPORTED[*]} — each with a 'compose' subcommand." >&2 + echo " Install one, or set RUNTIME= in .env if yours is under a different name." >&2 + return 1 + fi + fi + + # runc and its drop-in siblings get their own message: "unsupported runtime: runc" would read as an + # oversight, and the person would reasonably try to fix it by installing runc — which is very likely + # already installed, and still will not work. + case "$want" in + runc|crun|youki|gvisor|runsc|kata-runtime) + echo "ERROR: '$want' is a low-level OCI runtime, not a container CLI, so it cannot drive this kit." >&2 + echo " It runs an unpacked bundle handed to it by path — it has no images, registries," >&2 + echo " networks or compose, so there is no '$want run ' or '$want compose up'." >&2 + echo " It is what the CLI below uses underneath; to select it, tell that CLI:" >&2 + echo " RUNTIME=podman and podman --runtime $want … (or docker's default-runtime)" >&2 + echo " Set RUNTIME to one of: ${_RUNTIME_SUPPORTED[*]}" >&2 + return 1 ;; + esac + + local r ok=0 + for r in "${_RUNTIME_SUPPORTED[@]}"; do [ "$want" = "$r" ] && { ok=1; break; }; done + if [ "$ok" != 1 ]; then + echo "ERROR: RUNTIME='$want' is not a supported container CLI." >&2 + echo " Supported: ${_RUNTIME_SUPPORTED[*]}" >&2 + echo " (A runtime qualifies if it speaks Docker's CLI grammar and has a 'compose' subcommand.)" >&2 + return 1 + fi + + if ! command -v "$want" >/dev/null 2>&1; then + echo "ERROR: RUNTIME='$want' was requested but '$want' is not on PATH." >&2 + return 1 + fi + + RUNTIME="$want"; export RUNTIME + _RUNTIME_RESOLVED=1 + + # podman routes `podman compose` to an external provider (podman-compose or docker-compose) and prints + # a four-line banner about having done so, to stderr, on EVERY call. This kit makes many compose calls + # per run, and the banner is noise the user cannot act on. Only set the opt-out when it is unset, so + # anyone who genuinely wants the banner can keep it. + [ "$RUNTIME" = podman ] && : "${PODMAN_COMPOSE_WARNING_LOGS:=false}" && export PODMAN_COMPOSE_WARNING_LOGS + + return 0 +} + +# runtime_is_rootless_podman -> 1 or 0. +# +# Only podman can answer yes: docker's rootless mode still presents a daemon that owns the uid mapping, +# so the ownership problem below does not arise there. Cached because runtime_builder_ids is called on +# the hot path of every build and this shells out. +runtime_is_rootless_podman() { + if [ -z "${_RUNTIME_ROOTLESS:-}" ]; then + if [ "${RUNTIME:-docker}" = podman ] \ + && [ "$(podman info --format '{{.Host.Security.Rootless}}' 2>/dev/null)" = true ]; then + _RUNTIME_ROOTLESS=1 + else + _RUNTIME_ROOTLESS=0 + fi + fi + printf '%s' "$_RUNTIME_ROOTLESS" +} + +# runtime_builder_ids -> the " " pair to run the builder container as, so that the bundle it +# writes into the bind-mounted checkout ends up owned by the person who started the build. +# +# THE ANSWER IS NOT `id -u` EVERYWHERE, and this is the one place the runtimes genuinely diverge. +# +# Docker: the daemon runs as real root, so container uid N writes files owned by host uid N. Passing +# `id -u` is both correct and necessary — without it the bundle comes out root-owned. +# +# Rootless podman: the container is inside a user namespace where the host user is already mapped to +# uid 0, and the subuid range (/etc/subuid, typically 100000+) supplies every other id. So: +# - container uid 0 -> host uid 1000 (the caller — what we want) +# - container uid 1000 -> host uid 101000 (a subuid the caller cannot even write as) +# Passing `id -u` there is actively harmful: measured on podman 5.7, the build cannot create files in +# the mounted checkout at all, and anything it does create is owned by an id the caller cannot chown or +# delete without `podman unshare`. Asking for uid 0 is what yields caller-owned output. +# +# The alternative — `--userns=keep-id` with the real uid — needs a flag that has no Compose equivalent +# here, whereas this needs no change to docker-compose.yml at all. +runtime_builder_ids() { + if [ "$(runtime_is_rootless_podman)" = 1 ]; then + printf '0 0' + else + printf '%s %s' "$(id -u)" "$(id -g)" + fi +} + +# runtime_compose_project -> the compose project name these scripts' containers are labelled with. +# +# Compose's own default rule: the base directory name, lowercased, with everything outside [a-z0-9_-] +# dropped. COMPOSE_PROJECT_NAME overrides it, for both implementations. +runtime_compose_project() { + if [ -n "${COMPOSE_PROJECT_NAME:-}" ]; then printf '%s' "$COMPOSE_PROJECT_NAME"; return; fi + basename "$PWD" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9_-]//g' +} + +# runtime_compose_running_services -> this project's RUNNING compose services, one name per line. +# +# Empty output means "none running". A NON-ZERO RETURN means the question could not be answered at all, +# which callers must not conflate with "none" — see builder_studio_state, whose three-way answer exists +# for exactly this reason. +# +# Two implementations, because `compose ps` is where the CLIs diverge most. docker compose takes +# `--status running --services`; podman-compose 1.5 supports neither (its entire `ps` grammar is `-q` +# and `--format`) and exits 2 with a usage dump. Left as one call, that silently made switch-llm.sh +# conclude the agent was never running and skip the recreate — a wrong answer, not a visible failure. +# +# The fallback asks the RUNTIME rather than the compose wrapper: both implementations stamp the same +# com.docker.compose.{project,service} labels on every container they create (verified against +# podman-compose 1.5), and `ps --filter label=` with `{{.Label "…"}}` behaves identically in both CLIs. +# Trying the native form first means docker's behaviour is unchanged, down to the exit status. +runtime_compose_running_services() { + local out + if out="$("${RUNTIME:?}" compose ps --status running --services 2>/dev/null)"; then + printf '%s\n' "$out" + return 0 + fi + "$RUNTIME" ps --filter "label=com.docker.compose.project=$(runtime_compose_project)" \ + --format '{{.Label "com.docker.compose.service"}}' 2>/dev/null +} + +# runtime_label -> a human name for messages, e.g. "podman (rootless)". Purely cosmetic: a message that +# says "Docker" on a podman box sends people to debug the wrong thing. +runtime_label() { + if [ "$(runtime_is_rootless_podman)" = 1 ]; then printf 'podman (rootless)' + else printf '%s' "${RUNTIME:-docker}"; fi +} diff --git a/scripts/detect-bedrock.sh b/scripts/detect-bedrock.sh index 4081a4b..86d8de4 100755 --- a/scripts/detect-bedrock.sh +++ b/scripts/detect-bedrock.sh @@ -17,6 +17,8 @@ # permission rather than just the presence of a role. set -uo pipefail cd "$(dirname "$0")/.." +# shellcheck source=scripts/_runtime.sh +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman | nerdctl | finch) MODEL_ID="${1:-us.anthropic.claude-sonnet-5}" FORCE_REGION="${2:-}" @@ -181,17 +183,19 @@ printf '%s\n' "$OUT" # ok | blocked | unknown: # Only meaningful if the role itself checked out. if [ "$RC" = 0 ]; then - if ! command -v docker >/dev/null 2>&1; then - echo "CONTAINER_IMDS=unknown:docker not installed" - elif ! docker info >/dev/null 2>&1; then - echo "CONTAINER_IMDS=unknown:docker daemon not reachable" + # Quietly: this probe is one line of a diagnostic report, and a runtime that cannot be resolved is + # already reported as unknown: below rather than as a failure of the whole report. + if ! runtime_resolve 2>/dev/null; then + echo "CONTAINER_IMDS=unknown:no container runtime available" + elif ! "$RUNTIME" info >/dev/null 2>&1; then + echo "CONTAINER_IMDS=unknown:$RUNTIME not reachable" else - # The probe runs BEFORE `docker compose pull`, so on a fresh box this image is usually not local + # The probe runs BEFORE `$RUNTIME compose pull`, so on a fresh box this image is usually not local # yet. Pull it quietly rather than reporting a false "blocked". - docker image inspect busybox:1.36 >/dev/null 2>&1 || docker pull -q busybox:1.36 >/dev/null 2>&1 || true - if ! docker image inspect busybox:1.36 >/dev/null 2>&1; then + "$RUNTIME" image inspect busybox:1.36 >/dev/null 2>&1 || "$RUNTIME" pull -q busybox:1.36 >/dev/null 2>&1 || true + if ! "$RUNTIME" image inspect busybox:1.36 >/dev/null 2>&1; then echo "CONTAINER_IMDS=unknown:could not obtain busybox:1.36 (registry unreachable?)" - elif docker run --rm busybox:1.36 sh -c \ + elif "$RUNTIME" run --rm busybox:1.36 sh -c \ 'wget -S -T 3 -O /dev/null http://169.254.169.254/latest/meta-data/ 2>&1 | grep -q "HTTP/"' \ >/dev/null 2>&1; then echo "CONTAINER_IMDS=ok" diff --git a/scripts/switch-llm.sh b/scripts/switch-llm.sh index 9124c57..3174700 100755 --- a/scripts/switch-llm.sh +++ b/scripts/switch-llm.sh @@ -30,8 +30,10 @@ set -euo pipefail cd "$(dirname "$0")/.." ENV=.env [ -f "$ENV" ] || { echo "No .env found — run ./run.sh first to set up the stack." >&2; exit 1; } +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman | nerdctl | finch) . ./scripts/_provider_gateway.sh . ./scripts/_provider_subscription.sh +runtime_resolve || exit 1 NONINTERACTIVE=0 F_ANTHROPIC=""; F_AWS_KEY=""; F_AWS_SECRET=""; F_AWS_TOKEN=""; F_AWS_REGION="" @@ -221,13 +223,13 @@ esac setenv DEVKIT_MODEL "$TARGET" -if ! docker compose ps --status running --services 2>/dev/null | grep -qx claude-code-agent; then +if ! runtime_compose_running_services | grep -qx claude-code-agent; then echo "==> Agent container isn't running — .env is updated; start the stack with ./run.sh to apply it." >&2 exit 0 fi echo "==> Recreating the agent with the new provider…" -docker compose up -d claude-code-agent +"$RUNTIME" compose up -d claude-code-agent echo "==> Registering $(getenv CLAUDE_MODEL) ($LLM_DESC) as the System default LLM…" if LLM_PROVIDER_LABEL="$LLM_DESC" ./scripts/register-llm.sh; then diff --git a/stop.sh b/stop.sh index 47d7ff6..cd4f79f 100755 --- a/stop.sh +++ b/stop.sh @@ -2,11 +2,13 @@ # Stop the platform. Volumes are kept (loaded extensions + data survive). Pass --wipe to remove them. set -euo pipefail cd "$(dirname "$0")" +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman | nerdctl | finch) +runtime_resolve || exit 1 if [ "${1:-}" = "--wipe" ]; then echo "==> Stopping and REMOVING volumes (extensions, mongo, file store will be lost)…" - docker compose down -v + "$RUNTIME" compose down -v else echo "==> Stopping (volumes kept; extensions replay on next ./run.sh)…" - docker compose down + "$RUNTIME" compose down fi diff --git a/tests/test-runtime.sh b/tests/test-runtime.sh new file mode 100755 index 0000000..87112ea --- /dev/null +++ b/tests/test-runtime.sh @@ -0,0 +1,138 @@ +#!/usr/bin/env bash +# Checks the container-runtime abstraction in scripts/_runtime.sh, and that nothing in the kit has +# gone back to calling `docker` directly. +# +# Static + pure-function checks only: nothing here starts the stack, pulls an image, or needs any +# particular runtime to be installed — so it runs the same on a docker laptop, a podman box and CI. +set -uo pipefail +cd "$(dirname "$0")/.." || exit 1 +PASS=0; FAIL=0 +t() { printf ' %s … ' "$1"; } +ok() { echo "ok"; PASS=$((PASS+1)); } +bad() { echo "FAIL: $1"; FAIL=$((FAIL+1)); } + +# Resolution is exercised in a subshell per case: runtime_resolve exports RUNTIME and memoises itself, +# so cases would otherwise contaminate each other. DUPLO_ENV_FILE=/dev/null keeps a developer's real +# .env (which may pin RUNTIME) out of the auto-detect cases. +res() { # res -> prints "|" + ( eval "$1"; export DUPLO_ENV_FILE=/dev/null + . ./scripts/_runtime.sh + if runtime_resolve 2>/dev/null; then printf '0|%s' "$RUNTIME"; else printf '1|'; fi ) +} + +echo "container runtime abstraction:" + +t "_runtime.sh is syntactically valid and sourcing it defines the API" +if bash -n scripts/_runtime.sh 2>/dev/null && \ + ( . ./scripts/_runtime.sh + for f in runtime_detect runtime_resolve runtime_requested runtime_builder_ids \ + runtime_is_rootless_podman runtime_compose_project runtime_compose_running_services \ + runtime_label; do + declare -F "$f" >/dev/null || exit 1 + done ); then ok; else bad "missing function(s)"; fi + +t "every supported runtime is accepted when present on PATH" +MISS="" +for r in docker podman nerdctl finch; do + command -v "$r" >/dev/null 2>&1 || continue # only assert about what's installed here + [ "$(res "RUNTIME=$r")" = "0|$r" ] || MISS="$MISS $r" +done +if [ -z "$MISS" ]; then ok; else bad "rejected:$MISS"; fi + +t "runc and friends are rejected as low-level OCI runtimes, not silently accepted" +MISS="" +for r in runc crun youki runsc; do + [ "$(res "RUNTIME=$r")" = "1|" ] || MISS="$MISS $r" +done +if [ -z "$MISS" ]; then ok; else bad "not rejected:$MISS"; fi + +t "the runc rejection explains itself rather than saying 'unsupported'" +MSG="$( ( export DUPLO_ENV_FILE=/dev/null RUNTIME=runc + . ./scripts/_runtime.sh; runtime_resolve ) 2>&1 )" +if grep -q 'low-level OCI runtime' <<<"$MSG" && grep -q 'compose' <<<"$MSG" && grep -q 'podman' <<<"$MSG" +then ok; else bad "unhelpful message: $MSG"; fi + +t "an unknown runtime is rejected" +if [ "$(res "RUNTIME=nosuchruntime")" = "1|" ]; then ok; else bad "accepted"; fi + +t "a requested-but-absent runtime is rejected rather than falling back to another" +if command -v nerdctl >/dev/null 2>&1; then ok # can't test: it IS present +elif [ "$(res "RUNTIME=nerdctl")" = "1|" ]; then ok +else bad "fell back instead of failing"; fi + +t "values are cleaned of quotes, whitespace and CRs before use" +R="$(res 'RUNTIME=" podman "')" +if ! command -v podman >/dev/null 2>&1; then ok # nothing to assert on a podman-less box +elif [ "$R" = "0|podman" ]; then ok; else bad "got '$R'"; fi + +t "auto-detect picks an installed runtime, preferring docker" +R="$(res 'unset RUNTIME')" +EXPECT=""; for r in docker podman nerdctl finch; do command -v "$r" >/dev/null 2>&1 && { EXPECT="$r"; break; }; done +if [ -z "$EXPECT" ]; then ok # no runtime installed: nothing to prefer +elif [ "$R" = "0|$EXPECT" ]; then ok; else bad "got '$R', expected '0|$EXPECT'"; fi + +t "runtime_requested distinguishes an explicit choice from auto-detect" +A="$( ( export DUPLO_ENV_FILE=/dev/null; unset RUNTIME; . ./scripts/_runtime.sh; runtime_requested ) )" +B="$( ( export DUPLO_ENV_FILE=/dev/null RUNTIME=podman; . ./scripts/_runtime.sh; runtime_requested ) )" +if [ -z "$A" ] && [ "$B" = podman ]; then ok; else bad "auto='$A' explicit='$B'"; fi + +t "builder ids are the caller's uid/gid on docker, but 0:0 on rootless podman" +IDS="$( ( export DUPLO_ENV_FILE=/dev/null; . ./scripts/_runtime.sh; runtime_resolve 2>/dev/null + runtime_builder_ids ) )" +if [ -z "$IDS" ]; then bad "no ids" +elif [ "$( ( export DUPLO_ENV_FILE=/dev/null; . ./scripts/_runtime.sh + runtime_resolve 2>/dev/null; runtime_is_rootless_podman ) )" = 1 ]; then + [ "$IDS" = "0 0" ] && ok || bad "rootless podman should be '0 0', got '$IDS'" +else + [ "$IDS" = "$(id -u) $(id -g)" ] && ok || bad "expected '$(id -u) $(id -g)', got '$IDS'" +fi + +echo +echo "no stray hard-coded docker calls:" + +# The scripts that drive containers. Anything invoking the CLI must go through $RUNTIME so that the +# RUNTIME setting is actually honoured — a single stray `docker compose` silently ignores it. +# +# Detection strips comments AND double-quoted strings before matching: the kit legitimately NAMES these +# CLIs in help and error text ("Install docker, podman, …"), and only an actual invocation matters. A +# command name is never itself inside quotes, so dropping quoted spans cannot hide a real call. The +# `exec`/`command` prefix is matched explicitly — `exec docker compose` is exactly how logs.sh and the +# build dispatcher invoke it, and a check that missed that would pass while testing nothing. +t "no script invokes 'docker'/'podman' as a command instead of \$RUNTIME" +STRAY="" +for f in run.sh stop.sh logs.sh scripts/*.sh tests/*.sh; do + [ "$f" = tests/test-runtime.sh ] && continue # this file names them on purpose + sed -e 's/[[:space:]]*#.*$//' -e 's/"[^"]*"//g' "$f" \ + | grep -qE '(^|[^-[:alnum:]_./$])((exec|command)[[:space:]]+)?(docker|podman|nerdctl|finch)[[:space:]]+(compose|run|pull|build|images?|ps|info|version|rmi?|logs|volume|inspect|exec)\b' \ + && STRAY="$STRAY $f" +done +if [ -z "$STRAY" ]; then ok; else bad "hard-coded CLI in:$STRAY"; fi + +t "every script that runs containers sources _runtime.sh and resolves before using \$RUNTIME" +MISS="" +for f in run.sh stop.sh logs.sh scripts/_builder.sh scripts/switch-llm.sh scripts/detect-bedrock.sh; do + grep -q '_runtime.sh' "$f" || MISS="$MISS $f" +done +if [ -z "$MISS" ]; then ok; else bad "not sourced in:$MISS"; fi + +t "the builder run names its compose profile (podman-compose won't infer it)" +if grep -q 'compose --profile tools run' scripts/_builder.sh; then ok +else bad "scripts/_builder.sh must pass --profile tools"; fi + +t "docker-compose.yml has no nested \${...\${...}} (podman-compose mis-parses it)" +# Comments are stripped first: the comment above that line documents the broken form on purpose, and +# a comment cannot affect how compose interpolates anything. +if sed 's/[[:space:]]*#.*$//' docker-compose.yml | grep -qE '\$\{[^}]*\$\{'; then + bad "nested interpolation present"; else ok; fi + +t ".env.example documents RUNTIME and says runc is not a valid value" +if grep -q '^#RUNTIME=' .env.example && grep -qi 'runc' .env.example; then ok; else bad ".env.example"; fi + +t "bash -n on every shell script in the kit" +BADF="" +for f in run.sh stop.sh logs.sh scripts/*.sh tests/*.sh; do bash -n "$f" 2>/dev/null || BADF="$BADF $f"; done +if [ -z "$BADF" ]; then ok; else bad "syntax:$BADF"; fi + +echo +echo "$PASS passed, $FAIL failed" +[ "$FAIL" -eq 0 ] From 906bce029867e20c987030ff3ec2e29b78a34e41 Mon Sep 17 00:00:00 2001 From: Darren Paxton Date: Tue, 22 Sep 2026 10:04:08 -0400 Subject: [PATCH 02/12] feat(runtime): hard-fail on an undersized podman machine in run.sh MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit run.sh now refuses to start when the podman machine is too small to build in, instead of letting an undersized VM poison every later extension build. The failure this pre-empts is opaque: the VM's memory is fixed at create time and shared by the six-container stack AND every build, and the Angular Native-Federation build is what exhausts it first. The OOM kill surfaces as a bare `Killed`, ~100 lines of Go "all goroutines are asleep - deadlock!" from esbuild, and finally `exit status 137` — the only part that means "out of memory", and the last line anyone reads. scripts/_runtime.sh runtime_machine_check, alongside the other runtime_* helpers. Reads the CONFIGURED size via `podman machine list --format json` (answers for a stopped machine too, and is the number `podman machine set` changes). Memory below the floor is a hard failure; a low CPU count only warns. Parses with python3, which run.sh already hard-requires — jq would add an undeclared dependency to the scripts that source this file. Every read or parse failure returns 0: this exists to give a better message than exit 137, never to become a new way for the kit to refuse to start. No-op on docker, nerdctl/finch, and machine-less Linux podman. run.sh calls it after the runtime is resolved. .env.example documents PODMAN_MIN_MEMORY_MIB / PODMAN_MIN_CPUS. The 6 GiB floor is a judgement call, not a measurement, and the code says so: 2 GiB reliably OOMs the frontend build, 8 GiB completes comfortably, nothing between was tested. The remediation command derives its values from max(current, floor), so a raised floor is honoured and a 6-CPU machine short on memory is never told to downgrade itself. Co-Authored-By: Claude Opus 5 (1M context) --- .env.example | 9 +++ run.sh | 6 ++ scripts/_runtime.sh | 132 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 147 insertions(+) diff --git a/.env.example b/.env.example index cea4aa0..d13188b 100644 --- a/.env.example +++ b/.env.example @@ -58,6 +58,15 @@ BUILDER_TAG=latest # same thing for a single run, and wins over this. Leave it commented out to build in the container. #DUPLO_BUILD_NATIVE=1 +# Minimum podman VM size run.sh will start on (macOS/Windows only — a native-Linux podman has no VM, +# and docker/nerdctl/finch are not checked). The VM's memory is fixed at create time and shared by this +# stack AND every extension build; too little and the Angular frontend build is OOM-killed, reporting +# itself only as 'exit status 137' at the foot of a Go stack trace. Undersized memory is a hard failure; +# a low CPU count is just a warning. Lower the floor only if you know a smaller machine suits you — +# 2 GiB is known to fail, 8 GiB is comfortable. +#PODMAN_MIN_MEMORY_MIB=6144 +#PODMAN_MIN_CPUS=4 + # On Apple Silicon the studio image is amd64-only (emulated). Leave as-is unless you have an arm64 image. STUDIO_PLATFORM=linux/amd64 diff --git a/run.sh b/run.sh index beca595..d1bd6b0 100755 --- a/run.sh +++ b/run.sh @@ -145,6 +145,12 @@ fi [ "$RUNTIME_OK" = 1 ] || exit 1 "$RUNTIME" info >/dev/null 2>&1 || echo "Note: $RUNTIME doesn't look ready (daemon not running, or no connection) — sort that out before this gets to 'Pulling images'." >&2 +# Sized-VM check. Separate from the MISSING one-pass above because it needs RUNTIME already resolved, +# and it is a hard failure by design: an undersized podman machine does not stop the stack from coming +# up, it silently poisons every extension build later with an OOM kill that reports itself as a Go +# deadlock and 'exit status 137'. A no-op on docker, on nerdctl/finch, and on a machine-less podman. +runtime_machine_check || exit 1 + # ── .env helpers (line-based; safe for tokens/keys with special chars) ──────── [ -f "$ENV" ] || { [ -f .env.example ] && cp .env.example "$ENV" || touch "$ENV"; } getenv() { grep -E "^$1=" "$ENV" 2>/dev/null | head -1 | cut -d= -f2- || true; } diff --git a/scripts/_runtime.sh b/scripts/_runtime.sh index 050ee41..80312f9 100644 --- a/scripts/_runtime.sh +++ b/scripts/_runtime.sh @@ -217,3 +217,135 @@ runtime_label() { if [ "$(runtime_is_rootless_podman)" = 1 ]; then printf 'podman (rootless)' else printf '%s' "${RUNTIME:-docker}"; fi } + +# ── podman machine sizing ───────────────────────────────────────────────────── +# On macOS and Windows podman runs containers inside a VM whose memory is fixed at machine-create time +# and is NOT elastic the way Docker Desktop's is. That one ceiling is shared by the whole stack (six +# containers) AND by every extension build — and the Angular Native-Federation build is the hungriest +# thing the kit runs: esbuild bundling the federation artifacts is what hits the wall first. +# +# It fails badly enough to be worth pre-empting. The OOM kill surfaces as a bare `Killed`, then ~100 +# lines of Go "fatal error: all goroutines are asleep - deadlock!" out of esbuild, and finally +# `exit status 137` — which is the only part that means "out of memory", and it is the last line anyone +# reads. Spending a few milliseconds here to say it in one sentence is a good trade. +# +# The floor is a JUDGEMENT CALL, not a measurement. The two known data points are: a 2 GiB machine with +# the stack up reliably OOMs the frontend build, and 8 GiB completes it comfortably. Nothing between was +# measured. 6 GiB is set as the hard floor (~4 GiB left for a build once the stack is running) and 8 GiB +# is what the message recommends. Both are overridable in .env for anyone whose workload disagrees. +_RUNTIME_MIN_MEMORY_MIB=6144 +_RUNTIME_MIN_CPUS=4 + +# runtime_machine_check -> 0 when there is nothing to complain about, 1 when the VM is too small to +# build in. Undersized memory is the failure; CPU count only earns a warning, since a slow build still +# produces a bundle. +# +# Scoped to podman deliberately. Docker's VM is managed by Docker Desktop, whose memory is a GUI setting +# this script cannot read; nerdctl/finch have their own lifecycle. A podman on native Linux has no +# machine at all (`machine list` yields []) and returns 0 — there is no VM to size. +# +# Reads the CONFIGURED size via `machine list`, not the live one via `podman info`: machine list answers +# for a STOPPED machine too (so the check still fires on the very run that would start it), and the +# configured number is the one `podman machine set` changes. +# +# python3, not jq: run.sh hard-requires python3 and never requires jq, and this file is sourced by +# run.sh, stop.sh and logs.sh. Any failure to read or parse returns 0 — this check exists to give a +# better message than exit 137, never to become a new way for the kit to refuse to start. +runtime_machine_check() { + [ "${RUNTIME:-}" = podman ] || return 0 + command -v podman >/dev/null 2>&1 || return 0 + command -v python3 >/dev/null 2>&1 || return 0 + + local min_mem min_cpus raw parsed name mib cpus running + min_mem="$(_runtime_clean "$(_runtime_envv PODMAN_MIN_MEMORY_MIB)")" + min_cpus="$(_runtime_clean "$(_runtime_envv PODMAN_MIN_CPUS)")" + # Fall back to the built-in floor on anything that isn't a plain positive integer, rather than letting + # a typo'd .env value silently disable the check (empty compares as smaller than everything). + case "$min_mem" in ''|*[!0-9]*) min_mem=$_RUNTIME_MIN_MEMORY_MIB ;; esac + case "$min_cpus" in ''|*[!0-9]*) min_cpus=$_RUNTIME_MIN_CPUS ;; esac + + raw="$(podman machine list --format json 2>/dev/null)" || return 0 + [ -n "$raw" ] || return 0 + + parsed="$(printf '%s' "$raw" | python3 -c ' +import json, sys +try: + ms = json.load(sys.stdin) +except Exception: + sys.exit(0) +if not isinstance(ms, list) or not ms: + sys.exit(0) + +# The machine this run will actually talk to: the running one, else the default, else whatever is first. +m = next((x for x in ms if x.get("Running")), None) \ + or next((x for x in ms if x.get("Default")), None) \ + or ms[0] + +try: + mem = int(str(m.get("Memory", "0")).strip() or 0) +except ValueError: + sys.exit(0) +if mem <= 0: + sys.exit(0) +# Current podman reports Memory as a byte count rendered as a STRING; some older builds reported MiB. +# A value below 1 MiB can only be the latter, so treat it as already-MiB rather than rounding to zero. +mib = mem // (1024 * 1024) if mem >= (1 << 20) else mem + +try: + cpus = int(m.get("CPUs") or 0) +except (TypeError, ValueError): + cpus = 0 + +print("%s\t%d\t%d\t%d" % (m.get("Name", "podman-machine-default"), mib, cpus, + 1 if m.get("Running") else 0)) +' 2>/dev/null)" || return 0 + [ -n "$parsed" ] || return 0 + + IFS=$'\t' read -r name mib cpus running <&2 + fi + + [ "$mib" -lt "$min_mem" ] || return 0 + + # What to actually suggest. 8 GiB is the comfortable figure, but a raised PODMAN_MIN_MEMORY_MIB must + # win — otherwise the message demands 16 GiB and then hands over a command that sets 8. + local rec_mem=8192 + [ "$min_mem" -gt "$rec_mem" ] && rec_mem="$min_mem" + + # `podman machine set` writes every flag it is given, so the suggested --cpus must never be LOWER than + # what the machine already has: a 6-CPU machine that is merely short on memory would otherwise be told + # to downgrade itself to the 4-CPU floor. Unknown CPU count (0) falls back to the floor. + local rec_cpus="$min_cpus" + [ "$cpus" -gt "$rec_cpus" ] && rec_cpus="$cpus" + + # The stop/set/start dance is spelled out because `podman machine set --memory` refuses to run against + # a started machine, and because stopping it takes the whole stack down with it — someone mid-session + # deserves to know that before they paste the command, not after. + cat >&2 < Date: Mon, 28 Sep 2026 18:05:04 -0400 Subject: [PATCH 03/12] feat(runtime): hard-fail on a podman machine without Rosetta MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On Apple Silicon an amd64 studio image runs emulated, and podman does NOT enable Rosetta by default — `[machine] rosetta = true` in containers.conf is required, and an absent file behaves exactly like an explicit `false`. Without it amd64 falls through to QEMU user-mode, which cannot run the studio's .NET runtime. Every layer of that failure lies. QEMU aborts inside MapControllers() with SIGABRT; the container still reports "Up" because the crash does not take PID 1 down promptly; /healthz never answers; and run.sh then spends ~4.5 minutes on a dot loop before exiting with "Login failed" — a message about credentials, for a problem that has nothing to do with them. runtime_rosetta_check replaces all of that with one accurate message. It is scoped to the only combination that matters, and is a no-op on docker (Docker Desktop ships Rosetta itself), on an amd64 host, and on an arm64 studio image. The probe reads the machine's binfmt_misc handler list, NOT `podman machine inspect`: inspect reports the flag the VM was last STARTED with and goes stale the moment containers.conf changes, whereas the handler either exists in the running kernel or it does not. A stopped or unreachable machine cannot be judged, so it returns "unknown" and the check stays silent — like runtime_machine_check, this exists to replace a baffling failure with a clear one, never to become a new way for the kit to refuse to start. THE REMEDIATION IS READ FROM THE EXISTING CONFIG, and that is not polish. The obvious advice — appending `[machine]\nrosetta = true` to containers.conf — is actively harmful on any machine that already has one. It creates a second `[machine]` table, TOML rejects the duplicate key, and podman then refuses to run at all: Failed to obtain podman configuration: parsing containers.conf: toml: line 11: Key 'machine' has already been defined. That is strictly worse than the missing Rosetta it was meant to fix. So _runtime_rosetta_conf_state reads the file podman will actually read (honouring CONTAINERS_CONF) and the message branches five ways: create it, add the section, add the key under the existing section, flip `false` to `true`, or — the common case for anyone who configured this once and later recreated their VM — "your config is already correct, the machine simply has not been restarted since". The impure parts (_runtime_host_arch, runtime_rosetta_active) are separate functions so the policy is testable with no podman, no VM and no Apple Silicon host; the suite still runs identically on a docker laptop and in CI. Sixteen new tests. Each guard was mutation-tested — removed, confirmed a named test fails, restored — because three of the message assertions were initially vacuous: under this suite's `set -o pipefail`, `cmd | grep -q` inverts, since grep -q exits on first match, SIGPIPEs the writer, and the pipeline reports the writer's failure. They capture into a variable now, with a comment so it is not reintroduced. Also in this commit: - finch is TEMPORARILY de-listed from _RUNTIME_SUPPORTED pending validation. It is no longer auto-detected and RUNTIME=finch is rejected. This is a claim-reduction, not a removal: every finch code path is retained and still correct, including builder_run_direct and the probe fix below. Re-enable by putting `finch` back in the array. - builder_probe_runtime asks for `{{json .Server}}` rather than `{{.Server.Version}}`. finch's server struct has no Version field, so the narrower template died in the template engine — indistinguishable from an unreachable runtime, which reported a healthy finch as "no container runtime is available" and refused the build. - builder_run_direct runs the builder image directly for CLIs whose `compose` cannot express the invocation, with the evidence for each of the three reasons recorded at the function. - nerdctl removed from the remaining user-facing lists, matching the earlier removal from _RUNTIME_SUPPORTED. - docs: configuration.md gains the Container runtime section it was missing despite promising every .env variable; prerequisites.md gains the Apple Silicon podman setup (Rosetta, and the 6144 MiB floor vs init's 2048 default); troubleshooting.md gains the full symptom-to-fix entry. All three record that auto-detection tests presence only, with no daemon check, so installing Docker silently takes precedence over an installed podman, and all three warn never to append a second [machine] table. Co-Authored-By: Claude Opus 5 (1M context) --- .env.example | 10 +- README.md | 2 +- docs/configuration.md | 40 ++++++ docs/getting-started/prerequisites.md | 38 +++++- docs/troubleshooting.md | 183 +++++++++++++++++++++++++- logs.sh | 2 +- run.sh | 15 ++- scripts/_builder.sh | 114 ++++++++++++++-- scripts/_runtime.sh | 180 ++++++++++++++++++++++++- scripts/detect-bedrock.sh | 2 +- scripts/switch-llm.sh | 2 +- stop.sh | 2 +- tests/test-runtime.sh | 147 ++++++++++++++++++++- 13 files changed, 698 insertions(+), 39 deletions(-) diff --git a/.env.example b/.env.example index d13188b..6eb4b14 100644 --- a/.env.example +++ b/.env.example @@ -25,10 +25,12 @@ QDRANT_API_KEY=devkit-local-qdrant # ───────────────────────────────────────────────────────────────────────────── # Container runtime — which CLI runs the stack and the builds. # -# Leave it commented out to auto-detect, which tries docker, podman, nerdctl, finch in that order (so -# a machine that has always had Docker keeps behaving exactly as before). Set it to pin one: +# Leave it commented out to auto-detect, which tries docker then podman, taking the first one +# INSTALLED (presence only — it is not checked for being reachable), so a machine that has always had +# Docker keeps behaving exactly as before. Installing Docker alongside podman therefore moves you onto +# Docker silently. Set it to pin one: # -# docker | podman | nerdctl | finch +# docker | podman # # The requirement is a Docker-compatible CLI that also has a `compose` subcommand — podman delegates # that to podman-compose or docker-compose, so install one of those alongside it. @@ -59,7 +61,7 @@ BUILDER_TAG=latest #DUPLO_BUILD_NATIVE=1 # Minimum podman VM size run.sh will start on (macOS/Windows only — a native-Linux podman has no VM, -# and docker/nerdctl/finch are not checked). The VM's memory is fixed at create time and shared by this +# and docker is not checked). The VM's memory is fixed at create time and shared by this # stack AND every extension build; too little and the Angular frontend build is OOM-killed, reporting # itself only as 'exit status 137' at the foot of a Go stack trace. Undersized memory is a hard failure; # a low CPU count is just a warning. Lower the floor only if you know a smaller machine suits you — diff --git a/README.md b/README.md index 9e9028f..7eb3a64 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ hot-loads into the running platform with no restart. ## Quick start -You need **a container runtime with Compose v2** (docker, podman, nerdctl or finch — set `RUNTIME` +You need **a container runtime with Compose v2** (docker or podman — set `RUNTIME` in `.env` to pin one, otherwise it is auto-detected), **Python 3**, and access to an LLM. `./run.sh` prompts for Anthropic, AWS Bedrock, or an Anthropic-compatible LLM gateway (OpenRouter, Bifrost, LiteLLM, …). diff --git a/docs/configuration.md b/docs/configuration.md index 30c5f2a..978d6b4 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -8,6 +8,7 @@ Everything the dev kit reads comes from `.env` in the repo root. Copy it from `. > [SUPPORT.md](../SUPPORT.md) before sharing any output. - [How `.env` is maintained](#how-env-is-maintained) +- [Container runtime](#container-runtime) - [Licensing](#licensing) - [Image tags](#image-tags) - [Host ports](#host-ports) @@ -27,6 +28,45 @@ Framework-shipped defaults — the image tags, the studio platform, and the lice your `.env` **only if you have not diverged from the previously-applied default**. A tag you pinned is always kept. The mechanism is described in [upgrading.md](upgrading.md#how-defaults-are-adopted). +## Container runtime + +| Variable | Default | Effect | +| --- | --- | --- | +| `RUNTIME` | *unset* — auto-detect | Which container CLI every script drives. Accepts `docker` or `podman`. | + +Docker is the default and needs no configuration. Left unset, the scripts auto-detect by walking the +supported CLIs in order and taking the first one found. + +Two things about that detection are worth knowing, because both have cost people time: + +- **It tests presence only — there is no daemon check.** A CLI that is installed but not running still + wins the detection. This is deliberate (`scripts/_runtime.sh`): conflating "installed" with "working" + would make an unstarted Docker Desktop silently fall through to another runtime, and a build would + then run somewhere you did not intend. +- **Order is precedence.** Installing Docker alongside an existing podman silently moves you onto Docker. + A correctly configured podman machine can be up and idle while `./run.sh` uses Docker instead. + +Check which one you will actually get, and pin it if you care: + +```bash +bash -c 'source scripts/_runtime.sh; echo "using: $(runtime_detect)"' +``` + +```bash +RUNTIME=podman # docker | podman +``` + +An explicitly requested runtime that turns out to be unusable is a **hard error**, not a fallback — +silently building with something other than what you asked for is never right. Auto-detection finding +nothing may still fall back to a native toolchain build. + +`RUNTIME` does not select a *low-level* OCI runtime. `runc`, `crun`, `youki` and `runsc` are not valid +values — they run an already-unpacked bundle by path and have no images, registries or compose. Choose +one through your CLI instead (`podman --runtime crun`). + +See [prerequisites](getting-started/prerequisites.md#01-a-container-runtime-with-compose-v2) for +install-time requirements, including the podman-specific setup on Apple Silicon. + ## Licensing The dev kit is licensed, and the studio will not serve without a license. `run.sh` handles this on first diff --git a/docs/getting-started/prerequisites.md b/docs/getting-started/prerequisites.md index fecffa5..87f165e 100644 --- a/docs/getting-started/prerequisites.md +++ b/docs/getting-started/prerequisites.md @@ -14,8 +14,8 @@ setup scripts use as their JSON and `.env` editor. ## 0.1 A container runtime, with Compose v2 -Docker is the default and needs no configuration. **podman**, **nerdctl** and **finch** are also -supported — Docker Desktop, Colima, Rancher Desktop, plain Docker Engine and rootless podman all work. +Docker is the default and needs no configuration. **podman** is also supported — Docker Desktop, +Colima, Rancher Desktop, plain Docker Engine and rootless podman all work. What matters is that the runtime is reachable and that its `compose` subcommand (two words) exists. 1. Check both at once — substitute your runtime for `docker`. @@ -37,16 +37,46 @@ What matters is that the runtime is reachable and that its `compose` subcommand 2. If you have more than one runtime installed, or want to pin one, set `RUNTIME` in `.env`: ```bash - RUNTIME=podman # docker | podman | nerdctl | finch + RUNTIME=podman # docker | podman ``` - Left unset, the scripts auto-detect, trying `docker`, `podman`, `nerdctl`, `finch` in that order. + Left unset, the scripts auto-detect, trying `docker` then `podman`, and taking the first one + **installed** — presence only, with no check that it is running. Installing Docker alongside podman + therefore moves you onto Docker silently. Confirm which you will get: + + ```bash + bash -c 'source scripts/_runtime.sh; echo "using: $(runtime_detect)"' + ``` > **podman:** install the compose provider too — podman shells out to one rather than implementing > compose itself (`sudo apt-get install podman-compose`, or `brew install podman-compose`). Rootless > podman needs nothing else; the build scripts handle the user-namespace uid mapping for you, so > extension bundles come out owned by you rather than by root. > +> **podman on Apple Silicon** needs two more things before the studio will start, because the studio +> image is amd64 on some tags and runs emulated: +> +> 1. **Enable Rosetta.** podman does **not** enable it by default, and without it amd64 binaries fall +> through to QEMU, which cannot run the studio's .NET runtime. Create +> `~/.config/containers/containers.conf` with `[machine] rosetta = true` before starting the machine +> (the key is read on every `podman machine start`, so an existing machine only needs a stop/start, +> not re-creating). **If that file already exists, edit its `[machine]` section — never append a +> second `[machine]` table, which is a TOML duplicate-key error that stops podman running at all.** +> 2. **Size the machine.** `podman machine init` defaults to 2048 MiB; `run.sh` hard-fails below +> 6144 MiB, because an undersized VM does not stop the stack coming up — it poisons later extension +> builds with an OOM kill that reports itself as `exit status 137`. Use +> `podman machine init --cpus 4 -m 8192 --disk-size 100`. +> +> Verify both before your first run: +> +> ```bash +> podman machine ssh 'ls /proc/sys/fs/binfmt_misc/' # want a `rosetta` entry, no `qemu-x86_64` +> podman machine list # want MEMORY >= 6144 MiB +> ``` +> +> If the studio hangs on startup anyway, see +> [troubleshooting](../troubleshooting.md#runsh-hangs-on-waiting-for-studio-apple-silicon). +> > **Not a runtime:** `runc` (and `crun`, `youki`, `runsc`). Those are low-level OCI runtimes that run an > already-unpacked bundle by path — they have no images, registries or compose, so they cannot drive > this kit. They are what the CLIs above use underneath; select one via your CLI, e.g. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 5662a3c..30be36d 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -33,7 +33,7 @@ This kit needs a couple of things that aren't here yet: **Why** — `run.sh` checks its two host prerequisites before it touches anything: `python3` (it is the `.env` editor and the JSON parser for every API call the setup makes) and a container runtime -(`docker`, `podman`, `nerdctl` or `finch` — auto-detected, or pinned with `RUNTIME` in `.env`), +(`docker` or `podman` — auto-detected, or pinned with `RUNTIME` in `.env`), including its `compose` v2 *subcommand* — the standalone `docker-compose` v1 binary does not satisfy it, and podman needs a compose provider installed alongside it (`podman-compose`). Every missing item is listed in one pass, so the list is the whole list. @@ -100,6 +100,187 @@ a `compose` command *by hand* under podman, and the errors do not say what is ac `host.docker.internal` works under podman: the `extra_hosts: host-gateway` entry in `docker-compose.yml` maps it, and podman additionally provides `host.containers.internal` for the same address. +### `./run.sh` hangs on "Waiting for studio" (Apple Silicon) + +> `run.sh` now catches this before it happens — `runtime_rosetta_check` in `scripts/_runtime.sh` stops +> the run with the fix below rather than letting it hang. You should only reach this entry if the check +> could not reach the machine to judge it (it stays silent rather than guessing), or if you are on a +> version of the kit that predates it. The diagnosis and fix are the same either way. + +**Symptom** — minutes of dots, then a failure that blames the wrong thing: + +``` +==> Waiting for studio at http://localhost:60031 ........................... +==> Minting permanent admin API token… +Login failed for you@example.com — check the admin email/password (./run.sh --reset to re-enter). +``` + +The studio container looks healthy — `podman ps` reports `Up`. Its log does not: + +``` +at Program.
$(String[] args) in /src/Duplo.ai.studio/Program.cs:line 255 +qemu: uncaught target signal 6 (Aborted) - core dumped +``` + +**Why** — feature-branch studio images are built amd64-only (`platform: ${STUDIO_PLATFORM:-linux/amd64}` +in `docker-compose.yml`), so on Apple Silicon the studio runs emulated. There are two emulators and only +one of them works: + +- **Rosetta** — used when the podman VM has the Rosetta share attached. Runs .NET correctly. +- **QEMU user-mode** — the fallback. Cannot run .NET's JIT/reflection; the process takes `SIGABRT` while + `MapControllers()` walks controller attributes at startup. + +**Rosetta is opt-in, and podman does not enable it for you.** Unless +`~/.config/containers/containers.conf` sets `[machine] rosetta = true`, the VM is started with no Rosetta +share and every amd64 binary falls through to QEMU. The key is read on every `podman machine start`, not +only at `init`. Measured on podman 6.1.2, reading `AppleHypervisor.Vfkit.Rosetta` from the machine's JSON: + +| `containers.conf` | resulting `Vfkit.Rosetta` | +| --- | --- | +| `[machine] rosetta = true` | `true` | +| *file absent* | `false` | +| `[machine] rosetta = false` | `false` | + +An absent file behaves exactly like an explicit `false`. Do not assume the default is on. + +Nothing tells you it is off, because it then fails **silently** twice over: + +1. Inside the VM, `rosetta-activation.service` runs `mount -t virtiofs rosetta /var/mnt`, fails with + `wrong fs type, bad option, bad superblock on rosetta`, and **exits 0 anyway**. `systemctl status` + reports the unit as `status=0/SUCCESS`. +2. The studio container stays `Up` after the .NET process aborts — the crash does not take PID 1 down + promptly — so `podman ps` shows nothing wrong. + +A second possible cause, **not observed here and not verified** — recorded because the strings are present +in the `vfkit` binary and it would produce identical symptoms even with the config set correctly: vfkit +degrades on its own with `Rosetta installation failed. Continuing without Rosetta.` and still exits 0, +which would happen if Rosetta were not registered on the host when the machine starts and vfkit probes +for it. The diagnosis below catches it either way; step 1 of the fix rules it out. + +With the studio dead, `/healthz` never answers and the 90 × 3s poll in `run.sh` spins for ~4.5 minutes +before falling through to a login failure that has nothing to do with your password. + +> **Do not trust the top-level `Rosetta` field** in `~/.config/containers/podman/machine/applehv/podman-machine-default.json`. +> It is not the field that decides, and it is routinely the *opposite* of the effective value. The one +> that matters is `AppleHypervisor.Vfkit.Rosetta`, which is what `podman machine inspect` reports. + +**Diagnose** — the decisive check is the binfmt handler list inside the VM: + +```bash +podman machine ssh 'ls /proc/sys/fs/binfmt_misc/' +``` + +| You see | Meaning | +| --- | --- | +| `rosetta` present | Rosetta is live. This is not your problem — look elsewhere. | +| `qemu-x86_64` present, no `rosetta` | Broken. amd64 runs under QEMU and .NET will abort. | + +Corroborate: + +```bash +podman machine inspect --format '{{.Rosetta}}' # false when broken +podman machine ssh 'mount | grep -i rosetta' # no output when broken +podman machine ssh 'systemctl status rosetta-activation.service --no-pager -l' + # "wrong fs type" despite status=0/SUCCESS +``` + +**Fix** — add the config, then restart the machine. **The machine does not need to be recreated.** podman +reads `[machine] rosetta` at every `podman machine start`, not only at `init`, so a stop/start picks it up +and nothing is destroyed. + +1. Confirm Rosetta on the host first — the VM share cannot be created if the host has no Rosetta: + +```bash +arch -x86_64 /usr/bin/true && echo "Rosetta OK" || softwareupdate --install-rosetta --agree-to-license +``` + +If you just installed it, re-run the `arch -x86_64` check and see it succeed before going on. + +2. Opt in to Rosetta. **This is mandatory, not a nicety** — see the table above. + + **Only if `~/.config/containers/containers.conf` does not exist yet:** + +```bash +mkdir -p ~/.config/containers +printf '[machine]\nrosetta = true\n' > ~/.config/containers/containers.conf +``` + + > ⚠️ **If the file already exists, edit it — do not append.** Add `rosetta = true` under its + > existing `[machine]` section, or add a `[machine]` section if it has none. Appending a second + > `[machine]` table is a TOML duplicate-key error and podman then refuses to run **at all**: + > + > ``` + > Failed to obtain podman configuration: parsing containers.conf: + > toml: line 11: Key 'machine' has already been defined. + > ``` + > + > That is a worse state than the missing Rosetta you started with. `run.sh`'s preflight reads your + > actual config and prints the right instruction for whichever case you are in. + + Keep the file — it is read on every start, so deleting it silently loses Rosetta at the next one. + +3. Restart the machine. No `rm`, no `init`, no re-pull — images and volumes are untouched: + +```bash +podman machine stop +podman machine start +``` + +> Verified by toggling one machine twice: with the file absent a stop/start yields `Vfkit.Rosetta: false` +> and a `qemu-x86_64` binfmt handler; with the file present the same machine yields `true` and a `rosetta` +> handler. The machine's `Created` timestamp is unchanged throughout, so no re-creation is involved. +> +> Only recreate the machine if you need to change its **sizing** — and then pass your sizing back +> explicitly, because the `init` default is 2048 MiB and `runtime_machine_check` rejects anything under +> 6144: `podman machine init --cpus 4 -m 8192 --disk-size 100`. + +**Confirm** — run this gate *before* `./run.sh`, so a failure costs seconds rather than a confusing hang: + +```bash +podman machine inspect --format '{{.Rosetta}}' # true +podman machine ssh 'ls /proc/sys/fs/binfmt_misc/' | grep rosetta # rosetta +podman machine ssh 'mount | grep -i rosetta' # rosetta on /var/mnt type virtiofs +podman run --rm --platform linux/amd64 docker.io/library/alpine:3 uname -m # x86_64 +``` + +The first three must pass, and `qemu-x86_64` should now be **absent** from the binfmt list — Rosetta +replaces it. The fourth is a liveness check only: a trivial amd64 binary runs under QEMU too, so `x86_64` +there confirms amd64 executes but does **not** prove Rosetta is the thing executing it. The binfmt list is +the discriminating check. +Then start the stack and verify the studio is alive rather than merely `Up`: + +```bash +./run.sh +curl -fsS http://localhost:60031/healthz && echo " studio is answering" +podman logs devkit-duplo-ai-studio-1 2>&1 | grep -c 'uncaught target signal' # must be 0 +``` + +**Avoiding it entirely** — if an arm64 studio image exists for your tag, use it and skip emulation. +Observed so far: `branch-*` tags are published amd64-only, while `dev-*` tags are multi-arch. Never assume +either way — check: + +```bash +podman manifest inspect "quay.io/duplocloud/backend:$(grep '^STUDIO_TAG=' .env | cut -d= -f2)" \ + | grep architecture +``` + +If an `arm64` variant is listed, set `STUDIO_PLATFORM=linux/arm64` in `.env` and re-run `./run.sh`. + +**Or use Docker instead** — Docker Desktop on Apple Silicon provides Rosetta itself, with no +`containers.conf` and no machine configuration. The same amd64 studio image that aborts under podman's +QEMU starts normally under Docker: `uname -m` reports `x86_64` and the log carries zero +`uncaught target signal` entries. This whole entry is podman-specific. + +> **If you have both installed, check which one you are actually using.** `runtime_detect` in +> `scripts/_runtime.sh` walks `docker podman` and takes the first on `PATH` — presence only, with no +> daemon check — so installing Docker silently takes precedence over podman. A podman machine can be up +> and correctly configured while `./run.sh` uses Docker. Pin it deliberately: +> +> ```bash +> bash -c 'source scripts/_runtime.sh; echo "using: $(runtime_detect)"' # what it will pick +> echo 'RUNTIME=podman' >> .env # force podman +> ``` + ### More entries belong here Image pull failures and ` login quay.io`, and insufficient memory or disk. Add them as they are diff --git a/logs.sh b/logs.sh index bf9140f..5ac30d7 100755 --- a/logs.sh +++ b/logs.sh @@ -2,6 +2,6 @@ # Tail logs. Optional service name: duplo-ai-studio | claude-code-agent | duplo-ui | mongo | xterm set -euo pipefail cd "$(dirname "$0")" -. ./scripts/_runtime.sh # → $RUNTIME (docker | podman | nerdctl | finch) +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman) runtime_resolve || exit 1 exec "$RUNTIME" compose logs -f "$@" diff --git a/run.sh b/run.sh index d1bd6b0..6c6959d 100755 --- a/run.sh +++ b/run.sh @@ -50,7 +50,7 @@ set -euo pipefail cd "$(dirname "$0")" ENV=.env -. ./scripts/_runtime.sh # → $RUNTIME (docker | podman | nerdctl | finch); resolved in preflight +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman); resolved in preflight . ./scripts/_provider_gateway.sh . ./scripts/_provider_subscription.sh . ./scripts/_studio_api.sh @@ -135,8 +135,7 @@ if [ "$RUNTIME_OK" = 1 ] && ! "$RUNTIME" compose version >/dev/null 2>&1; then • $RUNTIME compose — '$RUNTIME compose version' failed. A v2-style compose subcommand is required; the standalone docker-compose v1 binary is not enough. docker — install the Compose v2 plugin, or upgrade Docker Desktop. - podman — install the provider it delegates to: apt install podman-compose (or dnf/brew). - nerdctl/finch — upgrade to a build that ships 'compose'." + podman — install the provider it delegates to: apt install podman-compose (or dnf/brew)." fi if [ -n "$MISSING" ]; then echo "This kit needs a couple of things that aren't here yet:$MISSING" >&2 @@ -148,9 +147,17 @@ fi # Sized-VM check. Separate from the MISSING one-pass above because it needs RUNTIME already resolved, # and it is a hard failure by design: an undersized podman machine does not stop the stack from coming # up, it silently poisons every extension build later with an OOM kill that reports itself as a Go -# deadlock and 'exit status 137'. A no-op on docker, on nerdctl/finch, and on a machine-less podman. +# deadlock and 'exit status 137'. A no-op on docker and on a machine-less podman. runtime_machine_check || exit 1 +# Rosetta preflight. Same shape and the same reason as the sized-VM check above: on Apple Silicon a +# podman machine without Rosetta hands amd64 binaries to QEMU, which cannot run the studio's .NET +# runtime — and every layer of that failure lies. The container reports "Up", /healthz simply never +# answers, and the wait loop further down burns 4.5 minutes before exiting on "Login failed", which +# points at credentials. Catching it here turns all of that into one accurate message. A no-op on +# docker, on an amd64 host, and on an arm64 studio image. +runtime_rosetta_check || exit 1 + # ── .env helpers (line-based; safe for tokens/keys with special chars) ──────── [ -f "$ENV" ] || { [ -f .env.example ] && cp .env.example "$ENV" || touch "$ENV"; } getenv() { grep -E "^$1=" "$ENV" 2>/dev/null | head -1 | cut -d= -f2- || true; } diff --git a/scripts/_builder.sh b/scripts/_builder.sh index 718ae4d..ebaa945 100644 --- a/scripts/_builder.sh +++ b/scripts/_builder.sh @@ -8,7 +8,7 @@ # is now the only prerequisite. # # Every container command here goes through $RUNTIME (see scripts/_runtime.sh, sourced below) rather -# than a literal `docker`, so the same build works under docker, podman, nerdctl or finch. +# than a literal `docker`, so the same build works under docker or podman. # # REQUIRES scripts/_target.sh, sourced FIRST: this file uses its `_envv` (to read .env for BUILDER_IMAGE / # BUILDER_PULL / BUILDER_TAG / STUDIO_PORT) and its `_TARGET`. Both current callers do source it first; @@ -109,15 +109,29 @@ builder_probe_runtime() { # `command -v` alone would answer 1 there and send the build looking for a socket that isn't mounted. # Only a runtime that actually answers counts. # - # `version --format {{.Server.Version}}` rather than `info`: printing the SERVER version requires - # contacting the server, so it proves exactly as much, and it is ~25x cheaper (measured 0.042s vs - # 1.074s). builder_dispatch evaluates all four probes as *arguments* to builder_mode, so bash runs - # this one even when --native has already decided the answer — and build-all.sh --native pays it once - # per extension. Podman is daemonless, but it implements the same query (reporting its own version as - # the server), so the one probe covers every supported runtime. + # `version --format` rather than `info`: rendering any part of the SERVER block requires contacting + # the server, so it proves exactly as much, and it is ~25x cheaper (measured 0.042s vs 1.074s). + # builder_dispatch evaluates all four probes as *arguments* to builder_mode, so bash runs this one + # even when --native has already decided the answer — and build-all.sh --native pays it once per + # extension. Podman is daemonless, but it implements the same query (reporting its own version as + # the server), so one probe covers every supported runtime. + # + # `{{json .Server}}` and NOT `{{.Server.Version}}`: the server struct is not the same shape across + # the supported CLIs. docker and podman put a version string at .Server.Version; finch (nerdctl + # underneath) does not have that field at all — its server block nests the component versions + # instead, so the narrower template dies in the template engine rather than on the wire: + # + # finch version --format '{{.Server.Version}}' + # fatal: template: version:1:9: executing "version" at <.Server.Version>: + # can't evaluate field Version in type main.NerdctlServerOutput + # + # That is indistinguishable from an unreachable runtime here, so a perfectly healthy finch was + # reported as "no container runtime is available" and the build refused to run. Asking for the whole + # object renders on all four and still fails when nothing answers — a stopped podman machine exits + # 125 (printing `null`) and a stopped Docker Desktop errors the same way, so the probe stays honest. [ -n "${RUNTIME:-}" ] || { echo 0; return; } command -v "$RUNTIME" >/dev/null 2>&1 || { echo 0; return; } - if "$RUNTIME" version --format '{{.Server.Version}}' >/dev/null 2>&1; then echo 1; else echo 0; fi + if "$RUNTIME" version --format '{{json .Server}}' >/dev/null 2>&1; then echo 1; else echo 0; fi } builder_missing_tools() { @@ -274,6 +288,71 @@ builder_studio_url() { printf '%s' "$url" } +# builder_run_direct [args...] +# +# The finch path: run the builder image directly rather than through `compose run`. Its compose +# implementation cannot express this invocation at all — three independent reasons, each verified against +# finch v1.19.0 (nerdctl v2.2.2 underneath), and the first two fail SILENTLY: +# +# 1. INTERPOLATION IGNORES THE SHELL ENVIRONMENT. It reads ${VAR} from the .env file only, so every value +# builder_dispatch exports falls through to its `:-` default instead. With DUPLO_BASE, DUPLO_UID/GID +# and BUILDER_IMAGE all exported, `finch compose config` still rendered `DUPLO_BASE: ""`, +# `user: 1000:1000` and the default image. An empty DUPLO_BASE then resolves in _target.sh to +# http://localhost:, which inside the container is the CONTAINER — so the SDK fetch dies +# on a URL that looks right; and uid 1000 writes the bundle as an id the caller may not own. +# 2. NO WAY TO SET THE IMAGE. `compose run` takes it from the compose file and has no --image flag, so +# together with (1) a pinned BUILDER_TAG/BUILDER_IMAGE has no route through at all. +# 3. A TTY IS FORCED. It always allocates one, and every spelling of the opt-out (--tty=false, -t, -T, +# --no-TTY) is rejected — so a caller whose stdout is not a console (CI, a script, an agent) dies on +# "provided file is not a console" before the build starts. +# +# `compose --env-file` would have covered 1 and 2 together; finch lists it in `compose --help` but +# mistranslates it into a `-e` that nerdctl rejects underneath, so it is not a way out either. +# +# EVERY FLAG BELOW MIRRORS THE `builder` SERVICE IN docker-compose.yml. That duplication is the price of +# this path — change the service and change this with it. +builder_run_direct() { + local script="$1"; shift + + # Seeded non-empty on purpose: macOS ships bash 3.2, where expanding an EMPTY array under `set -u` + # ("${args[@]}") is an unbound-variable error. + local args + args=(run --rm -w /work -u "$DUPLO_UID:$DUPLO_GID" -v "$PWD:/work") + + # The per-uid package caches, named exactly as the top-level `volumes:` block names them, so this path + # and the compose path warm the SAME caches rather than each paying for a cold build. + args+=(-v "duplo_devkit_nuget_$DUPLO_UID:/cache/nuget") + args+=(-v "duplo_devkit_npm_$DUPLO_UID:/cache/npm") + + # extra_hosts: the route to a studio that is not on this compose network. + args+=(--add-host "host.docker.internal:host-gateway") + + # Join the project network ONLY when the studio is in this project — that is exactly the case in which + # builder_studio_url chose the bare service name http://duplo-ai-studio:60021, which nothing off that + # network can resolve. When it is absent or unknown the URL is the published host port, which is + # reachable without joining anything (and the network may not even exist). + if [ "$(builder_studio_state)" = in-project ]; then + args+=(--net "$(runtime_compose_project)_default") + fi + + # platform: ${BUILDER_PLATFORM:-} — omitted entirely when unset, so the build stays on the host's + # native architecture instead of silently emulating one. + if [ -n "${BUILDER_PLATFORM:-}" ]; then + args+=(--platform "$BUILDER_PLATFORM") + fi + + # The environment: block, value for value. Passed as real flags, which is the whole point: unlike + # compose interpolation above, `run -e` does not consult .env and cannot silently drop them. + args+=(-e "DUPLO_BUILD_NATIVE=1") + args+=(-e "DUPLO_TARGET=${DUPLO_TARGET:-local}") + args+=(-e "DUPLO_BASE=${DUPLO_BASE:-}") + args+=(-e "DUPLO_HOST=${DUPLO_HOST:-}") + args+=(-e "DUPLO_TOKEN=${DUPLO_TOKEN:-}") + args+=(-e "DUPLO_ADMIN_TOKEN=${DUPLO_ADMIN_TOKEN:-}") + + exec "$RUNTIME" "${args[@]}" "$BUILDER_IMAGE" "$script" "$@" +} + # builder_dispatch [args...] # Returns 0 to mean "carry on natively". Otherwise it runs the build in the container and exits with # the container's status — it never returns in that case. @@ -322,7 +401,7 @@ builder_dispatch() { error:runtime) echo "ERROR: no container runtime is available and the local toolchain is incomplete." >&2 echo " Missing: $(builder_missing_tools)" >&2 - echo " Install docker, podman, nerdctl or finch (with a 'compose' subcommand) — that is the" >&2 + echo " Install docker or podman (with a 'compose' subcommand) — that is the" >&2 echo " only prerequisite — or install the toolchain yourself and re-run with --native." >&2 exit 1 ;; container) ;; @@ -376,12 +455,25 @@ builder_dispatch() { echo "==> Building in $BUILDER_IMAGE via $(runtime_label) (uid $DUPLO_UID:$DUPLO_GID)" \ "— --native to use your own toolchain" # --rm: the container is a one-shot. --no-deps: never start the platform as a side effect of a build. - # -T: nothing in a build wants a TTY, and this is exec'd from a script whose stdio may already be a pipe. # # --profile tools is REQUIRED, not belt-and-braces: the builder service sits behind that profile, and # while `docker compose run` implicitly enables the profiles of the service it was asked to run, # podman-compose 1.5 does not — it reports "missing services [builder]" and exits 1. Naming the # profile explicitly is a no-op on docker and the only way in on podman. (COMPOSE_PROFILES=tools is # NOT an alternative here: podman-compose ignores that variable.) - exec "$RUNTIME" compose --profile tools run --rm -T --no-deps builder "$script" "$@" + # + # -T (don't allocate a pseudo-TTY) is what makes this safe to exec from a script whose stdio is + # already a pipe — CI, or an agent. Nothing in a build wants a TTY. + # + # TWO DISPATCH PATHS, because `compose run` is only usable on half the supported CLIs. docker and + # podman keep the original invocation verbatim. finch cannot use compose for this at all: it rejects + # -T outright ("unknown shorthand flag: 'T' in -T") and then forces a TTY anyway with no opt-out, on + # top of ignoring the exported DUPLO_BASE/DUPLO_UID/BUILDER_IMAGE entirely — so it runs the image + # directly instead. builder_run_direct has the evidence for each of those. + case "$RUNTIME" in + docker|podman) + exec "$RUNTIME" compose --profile tools run --rm -T --no-deps builder "$script" "$@" ;; + *) + builder_run_direct "$script" "$@" ;; + esac } diff --git a/scripts/_runtime.sh b/scripts/_runtime.sh index 80312f9..c1d9e8a 100644 --- a/scripts/_runtime.sh +++ b/scripts/_runtime.sh @@ -11,12 +11,26 @@ # Every value below is a *client CLI* that speaks Docker's command grammar — `run`, `pull`, `build`, # `info`, `version`, `image inspect` — AND ships a `compose` subcommand, because this kit drives the # whole stack through Compose. That is the entire contract; anything meeting it can be added to -# _RUNTIME_SUPPORTED without touching another line. +# _RUNTIME_SUPPORTED — but check the dispatch split in scripts/_builder.sh first, because `compose run` +# is not usable on every CLI that meets the contract, and the fallback branch there assumes a +# finch-shaped CLI. # # docker the default, and what the kit was written against. # podman daemonless; rootless by default. See the uid note on runtime_builder_ids below. -# nerdctl containerd's CLI. -# finch AWS's macOS/Windows wrapper around nerdctl in a VM. +# +# finch is TEMPORARILY de-listed, pending validation — it is not auto-detected and RUNTIME=finch is +# rejected. This is a claim-reduction, not a removal: every finch code path is still here and still +# correct, including the compose-bypass dispatch in scripts/_builder.sh and its notes on the three +# ways finch's `compose` fails silently. Re-enable by putting `finch` back in the array below; nothing +# else needs touching. The reason for de-listing is the same one that removed nerdctl: the kit should +# not advertise a runtime nobody is currently exercising. +# +# nerdctl is deliberately NOT here, and is not coming back. finch is the supported way to drive +# containerd in this kit: it is nerdctl plus the VM and the defaults that make it work on macOS and +# Windows, which is where anyone here is running containerd in the first place. Bare nerdctl was listed +# once and never exercised, and it shares finch's compose limitations without finch's setup. A +# native-Linux containerd user can still pin RUNTIME to a CLI of their choice by adding it back here — +# see the note above about the builder's dispatch split. # # runc is deliberately NOT here, though it was asked for. It is a *low-level OCI runtime*: it executes # an already-unpacked bundle (a rootfs plus config.json) given by path, and has no concept of an image, @@ -28,7 +42,12 @@ # The order here is also the auto-detect precedence: docker first, so a machine that has always had # Docker keeps behaving exactly as it did before this file existed. -_RUNTIME_SUPPORTED=(docker podman nerdctl finch) +# +# NOTE: precedence is by PRESENCE, not by liveness (see runtime_detect) — so an installed-but-stopped +# docker still wins over a running podman, and the build then fails rather than falling through. Pin +# RUNTIME in .env on a machine that has both. Installing Docker alongside an existing podman is the +# common way to land on the wrong one without noticing. +_RUNTIME_SUPPORTED=(docker podman) # Self-contained .env read. _target.sh defines an identical `_envv`, but this file is sourced by run.sh # and stop.sh too, which never source _target.sh — and it must work before either has run. Same @@ -241,7 +260,7 @@ _RUNTIME_MIN_CPUS=4 # produces a bundle. # # Scoped to podman deliberately. Docker's VM is managed by Docker Desktop, whose memory is a GUI setting -# this script cannot read; nerdctl/finch have their own lifecycle. A podman on native Linux has no +# this script cannot read. A podman on native Linux has no # machine at all (`machine list` yields []) and returns 0 — there is no VM to size. # # Reads the CONFIGURED size via `machine list`, not the live one via `podman info`: machine list answers @@ -349,3 +368,154 @@ PODMAN_MIN_MEMORY_MIB in .env to lower the floor. MSG return 1 } + +# Normalised host CPU architecture: amd64 | arm64 | whatever uname said. Its own function so the rosetta +# preflight's policy can be exercised from a test on any host. +_runtime_host_arch() { + local m; m="$(uname -m 2>/dev/null)" + case "$m" in + x86_64|amd64) printf 'amd64' ;; + arm64|aarch64) printf 'arm64' ;; + *) printf '%s' "$m" ;; + esac +} + +# runtime_rosetta_active -> 0 Rosetta is live in the podman machine, 1 definitively not, 2 could not tell. +# +# The binfmt handler list is the ONLY honest source. `podman machine inspect` reports the flag the VM was +# last STARTED with, which goes stale the moment containers.conf changes; the handler either exists in the +# running kernel or it does not. The cost is that a stopped or unreachable machine cannot be judged at all +# — hence 2, which the caller treats as "say nothing". +runtime_rosetta_active() { + command -v podman >/dev/null 2>&1 || return 2 + local handlers + handlers="$(podman machine ssh 'ls /proc/sys/fs/binfmt_misc/' 2>/dev/null)" || return 2 + [ -n "$handlers" ] || return 2 + printf '%s\n' "$handlers" | grep -qx 'rosetta' && return 0 + return 1 +} + +# _runtime_rosetta_conf_state -> absent | no-machine | no-key | disabled | enabled +# +# What containers.conf currently says about Rosetta, so the remediation can be specific. It has to be, +# because the obvious advice is actively harmful: appending a second `[machine]` table to a file that +# already has one is a TOML duplicate-key error, and podman then refuses to do ANYTHING — +# +# Failed to obtain podman configuration: parsing containers.conf: toml: line 11: +# Key 'machine' has already been defined. +# +# — which is a worse place to be than the missing Rosetta we were trying to fix. The `enabled` state +# matters just as much: config that is already right means the machine simply has not been restarted +# since, and telling someone to re-add a key they already have sends them looking in the wrong place. +# +# CONTAINERS_CONF is podman's own override for this path, so honouring it keeps us reading the same +# file podman will, and lets the tests point at a fixture. +_runtime_rosetta_conf_state() { + local f="${CONTAINERS_CONF:-$HOME/.config/containers/containers.conf}" + [ -f "$f" ] || { printf 'absent'; return; } + # The [machine] table runs from its header to the next [section] header or EOF. + local block + block="$(awk '/^[[:space:]]*\[machine\][[:space:]]*$/{f=1;next} /^[[:space:]]*\[/{f=0} f' "$f" 2>/dev/null)" + [ -n "$block" ] || { grep -qE '^[[:space:]]*\[machine\][[:space:]]*$' "$f" 2>/dev/null \ + && { printf 'no-key'; return; }; printf 'no-machine'; return; } + case "$(printf '%s' "$block" | grep -E '^[[:space:]]*rosetta[[:space:]]*=' | tail -1)" in + *true*) printf 'enabled' ;; + *false*) printf 'disabled' ;; + *) printf 'no-key' ;; + esac +} + +# runtime_rosetta_check -> 0 when there is nothing to complain about, 1 when an amd64 studio image is +# about to be handed to QEMU on an Apple Silicon host. +# +# Why this is a hard failure and not a note: QEMU user-mode cannot run the studio's .NET runtime. It +# aborts inside MapControllers() with SIGABRT, the container stays "Up" because the crash does not take +# PID 1 down promptly, /healthz never answers, and run.sh then spends 4.5 minutes on a dot loop before +# failing with "Login failed" — a message about credentials, for a problem that has nothing to do with +# them. Every layer of that is silent or misleading, so it is caught here instead. +# +# Scoped tightly, because Rosetta is only ever relevant to one combination: +# - docker is not checked: Docker Desktop provides Rosetta itself, with nothing to configure. +# - a non-amd64 studio platform needs no translation at all. +# - a non-arm64 host needs no translation to run amd64. +# Anything it cannot determine returns 0. Like runtime_machine_check, this exists to replace a baffling +# failure with a clear one, never to become a new way for the kit to refuse to start. +runtime_rosetta_check() { + [ "${RUNTIME:-}" = podman ] || return 0 + + local plat host rc + # Same precedence as runtime_requested: environment wins over .env. The fallback matches the compose + # default (`platform: ${STUDIO_PLATFORM:-linux/amd64}`), so an unset value is amd64 here too. + plat="$(_runtime_clean "${STUDIO_PLATFORM:-$(_runtime_envv STUDIO_PLATFORM)}")" + [ -n "$plat" ] || plat=linux/amd64 + case "$plat" in *amd64*|*x86_64*) ;; *) return 0 ;; esac + + host="$(_runtime_host_arch)" + [ "$host" = arm64 ] || return 0 + + runtime_rosetta_active; rc=$? + [ "$rc" = 1 ] || return 0 + + # The remediation depends on what containers.conf already says. This is not polish: telling someone to + # append `[machine]` to a file that already has that table produces a TOML duplicate-key error, and + # podman then refuses to run at all ("Key 'machine' has already been defined") — strictly worse than + # the missing Rosetta. And when the key is already correct, the file is not the problem: the machine + # simply has not been restarted since it was set. + local conf_path conf_dir fix + conf_path="${CONTAINERS_CONF:-$HOME/.config/containers/containers.conf}" + conf_dir="$(dirname "$conf_path")" + case "$(_runtime_rosetta_conf_state)" in + absent) + fix="No $conf_path yet — create it: + + mkdir -p $conf_dir + printf '[machine]\\nrosetta = true\\n' > $conf_path" ;; + no-machine) + fix="$conf_path exists but has no [machine] section. Open it and ADD this section +(do not append a second one elsewhere in the file): + + [machine] + rosetta = true" ;; + no-key) + fix="$conf_path already has a [machine] section. Add this line UNDER that existing +section — do not add a second [machine] header, TOML rejects a duplicate table and podman +then refuses to start at all: + + rosetta = true" ;; + disabled) + fix="$conf_path sets 'rosetta = false' under [machine]. Change that one value to: + + rosetta = true" ;; + enabled) + fix="$conf_path ALREADY sets 'rosetta = true' — the file is correct and needs no edit. +The key is read at every machine start, so the machine simply has not been restarted since." ;; + esac + + cat >&2 <&2; exit 1; } -. ./scripts/_runtime.sh # → $RUNTIME (docker | podman | nerdctl | finch) +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman) . ./scripts/_provider_gateway.sh . ./scripts/_provider_subscription.sh runtime_resolve || exit 1 diff --git a/stop.sh b/stop.sh index cd4f79f..903b6f1 100755 --- a/stop.sh +++ b/stop.sh @@ -2,7 +2,7 @@ # Stop the platform. Volumes are kept (loaded extensions + data survive). Pass --wipe to remove them. set -euo pipefail cd "$(dirname "$0")" -. ./scripts/_runtime.sh # → $RUNTIME (docker | podman | nerdctl | finch) +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman) runtime_resolve || exit 1 if [ "${1:-}" = "--wipe" ]; then diff --git a/tests/test-runtime.sh b/tests/test-runtime.sh index 87112ea..5c91ba0 100755 --- a/tests/test-runtime.sh +++ b/tests/test-runtime.sh @@ -27,13 +27,13 @@ if bash -n scripts/_runtime.sh 2>/dev/null && \ ( . ./scripts/_runtime.sh for f in runtime_detect runtime_resolve runtime_requested runtime_builder_ids \ runtime_is_rootless_podman runtime_compose_project runtime_compose_running_services \ - runtime_label; do + runtime_label runtime_rosetta_check; do declare -F "$f" >/dev/null || exit 1 done ); then ok; else bad "missing function(s)"; fi t "every supported runtime is accepted when present on PATH" MISS="" -for r in docker podman nerdctl finch; do +for r in docker podman; do command -v "$r" >/dev/null 2>&1 || continue # only assert about what's installed here [ "$(res "RUNTIME=$r")" = "0|$r" ] || MISS="$MISS $r" done @@ -56,8 +56,15 @@ t "an unknown runtime is rejected" if [ "$(res "RUNTIME=nosuchruntime")" = "1|" ]; then ok; else bad "accepted"; fi t "a requested-but-absent runtime is rejected rather than falling back to another" -if command -v nerdctl >/dev/null 2>&1; then ok # can't test: it IS present -elif [ "$(res "RUNTIME=nerdctl")" = "1|" ]; then ok +# Must name a SUPPORTED runtime that happens not to be installed here — an unsupported name would be +# rejected by the wrong branch and the test would pass while proving nothing. Which one that is differs +# per machine, so find it rather than hard-coding one. +ABSENT="" +for r in docker podman; do + command -v "$r" >/dev/null 2>&1 || { ABSENT="$r"; break; } +done +if [ -z "$ABSENT" ]; then ok # every supported runtime is installed here +elif [ "$(res "RUNTIME=$ABSENT")" = "1|" ]; then ok else bad "fell back instead of failing"; fi t "values are cleaned of quotes, whitespace and CRs before use" @@ -67,7 +74,7 @@ elif [ "$R" = "0|podman" ]; then ok; else bad "got '$R'"; fi t "auto-detect picks an installed runtime, preferring docker" R="$(res 'unset RUNTIME')" -EXPECT=""; for r in docker podman nerdctl finch; do command -v "$r" >/dev/null 2>&1 && { EXPECT="$r"; break; }; done +EXPECT=""; for r in docker podman; do command -v "$r" >/dev/null 2>&1 && { EXPECT="$r"; break; }; done if [ -z "$EXPECT" ]; then ok # no runtime installed: nothing to prefer elif [ "$R" = "0|$EXPECT" ]; then ok; else bad "got '$R', expected '0|$EXPECT'"; fi @@ -76,6 +83,126 @@ A="$( ( export DUPLO_ENV_FILE=/dev/null; unset RUNTIME; . ./scripts/_runtime.sh; B="$( ( export DUPLO_ENV_FILE=/dev/null RUNTIME=podman; . ./scripts/_runtime.sh; runtime_requested ) )" if [ -z "$A" ] && [ "$B" = podman ]; then ok; else bad "auto='$A' explicit='$B'"; fi +# Rosetta preflight. The two impure parts — what the host CPU is, and whether the podman machine has a +# rosetta binfmt handler — are separate functions precisely so the POLICY can be exercised here without +# podman, a VM, or an Apple Silicon host. Redefining them after sourcing is the seam. +ros() { # ros [runtime] -> "|" + ( HA="$1"; PL="$2"; PRC="$3" + export DUPLO_ENV_FILE=/dev/null RUNTIME="${4:-podman}" STUDIO_PLATFORM="$PL" + . ./scripts/_runtime.sh + eval "_runtime_host_arch() { printf '%s' '$HA'; }" + eval "runtime_rosetta_active() { return $PRC; }" + ERR="$(runtime_rosetta_check 2>&1 >/dev/null)"; RC=$? + printf '%s|%s' "$RC" "$(printf '%s' "$ERR" | head -1)" ) +} + +# The remediation the check prints depends on what containers.conf already says. Appending a second +# [machine] table to an existing file is a TOML "key already defined" error that stops podman dead, so +# the advice must never be one-size-fits-all. CONTAINERS_CONF is podman's own override, so pointing it +# at a fixture is both faithful and testable. +conf() { # conf -> the state word + ( F=/dev/null + if [ "$1" != NONE ]; then F="$(mktemp)"; printf '%s' "$1" > "$F"; else F="$(mktemp -u)"; fi + export CONTAINERS_CONF="$F" + . ./scripts/_runtime.sh + _runtime_rosetta_conf_state ) +} + +t "conf state reports absent when containers.conf does not exist" +[ "$(conf NONE)" = absent ] && ok || bad "got '$(conf NONE)'" + +t "conf state reports enabled when [machine] already sets rosetta = true" +R="$(conf '[containers] +[machine] +rosetta = true +[network] +')" +[ "$R" = enabled ] && ok || bad "got '$R'" + +t "conf state reports disabled when [machine] explicitly sets rosetta = false" +R="$(conf '[machine] +rosetta = false +')" +[ "$R" = disabled ] && ok || bad "got '$R'" + +t "conf state reports no-key when [machine] exists without a rosetta key" +R="$(conf '[containers] +[machine] +cpus = 4 +')" +[ "$R" = no-key ] && ok || bad "got '$R'" + +t "conf state reports no-machine when the file exists with no [machine] section" +R="$(conf '[containers] +[engine] +env = [] +')" +[ "$R" = no-machine ] && ok || bad "got '$R'" + +t "rosetta check fails when an amd64 image would run on an arm64 host without Rosetta" +R="$(ros arm64 linux/amd64 1)" +case "$R" in 1\|*[Rr]osetta*) ok ;; *) bad "got '$R'" ;; esac + +rosmsg() { # rosmsg -> the whole stderr message + ( C="$1" + if [ "$C" != NONE ]; then F="$(mktemp)"; printf '%s' "$C" > "$F"; else F="$(mktemp -u)"; fi + export DUPLO_ENV_FILE=/dev/null RUNTIME=podman STUDIO_PLATFORM=linux/amd64 CONTAINERS_CONF="$F" + . ./scripts/_runtime.sh + _runtime_host_arch() { printf 'arm64'; } + runtime_rosetta_active() { return 1; } + runtime_rosetta_check 2>&1 >/dev/null ) +} + +# NOTE: capture into a variable, never `rosmsg ... | grep -q`. Under the `set -o pipefail` at the top of +# this file, grep -q exits on first match, SIGPIPEs the writer, and the pipeline reports the writer's +# failure — so the condition reads FALSE precisely when the pattern matched. That silently inverted +# these three assertions when they were first written. +t "the fix never suggests appending a second [machine] table to an existing config" +M="$(rosmsg '[containers] +[machine] +rosetta = false +')" +if printf '%s\n' "$M" | grep -qE '>>[[:space:]]*[^|]*containers\.conf'; then + bad "tells the user to append a second [machine] table — a TOML duplicate-key error that stops podman" +else ok; fi + +t "a config that already enables rosetta points at a restart, not at editing the file again" +M="$(rosmsg '[machine] +rosetta = true +')" +if printf '%s\n' "$M" | grep -qi 'already' && printf '%s\n' "$M" | grep -qi 'podman machine stop'; then ok +else bad "got: $(printf '%s' "$M" | sed -n '9,12p' | tr '\n' ' ')"; fi + +t "a missing config gets a create command that cannot clobber anything" +M="$(rosmsg NONE)" +# `>` not `>>`: there is no file to append to, and `>>` would be the wrong habit to teach. +if printf '%s\n' "$M" | grep -qE "printf .* > " && printf '%s\n' "$M" | grep -qi 'create it'; then ok +else bad "no safe create command: $(printf '%s' "$M" | sed -n '10,13p' | tr '\n' ' ')"; fi + +t "rosetta check is a no-op on docker, which provides Rosetta itself" +R="$(ros arm64 linux/amd64 1 docker)" +[ "$R" = "0|" ] && ok || bad "got '$R'" + +t "rosetta check is a no-op for an arm64 studio image, which needs no translation" +R="$(ros arm64 linux/arm64 1)" +[ "$R" = "0|" ] && ok || bad "got '$R'" + +t "rosetta check is a no-op on an amd64 host, which needs no translation" +R="$(ros amd64 linux/amd64 1)" +[ "$R" = "0|" ] && ok || bad "got '$R'" + +t "rosetta check is a no-op when Rosetta is active" +R="$(ros arm64 linux/amd64 0)" +[ "$R" = "0|" ] && ok || bad "got '$R'" + +t "rosetta check stays quiet when the machine cannot be probed rather than crying wolf" +R="$(ros arm64 linux/amd64 2)" +[ "$R" = "0|" ] && ok || bad "got '$R'" + +t "rosetta check treats an unset STUDIO_PLATFORM as amd64, matching the compose default" +R="$(ros arm64 "" 1)" +case "$R" in 1\|*[Rr]osetta*) ok ;; *) bad "got '$R'" ;; esac + t "builder ids are the caller's uid/gid on docker, but 0:0 on rootless podman" IDS="$( ( export DUPLO_ENV_FILE=/dev/null; . ./scripts/_runtime.sh; runtime_resolve 2>/dev/null runtime_builder_ids ) )" @@ -98,6 +225,9 @@ echo "no stray hard-coded docker calls:" # command name is never itself inside quotes, so dropping quoted spans cannot hide a real call. The # `exec`/`command` prefix is matched explicitly — `exec docker compose` is exactly how logs.sh and the # build dispatcher invoke it, and a check that missed that would pass while testing nothing. +# +# nerdctl and finch stay in the alternation below even though neither is a SUPPORTED runtime: this is a +# lint against hard-coding any CLI, and hard-coding an unsupported one is worse, not better. t "no script invokes 'docker'/'podman' as a command instead of \$RUNTIME" STRAY="" for f in run.sh stop.sh logs.sh scripts/*.sh tests/*.sh; do @@ -108,6 +238,13 @@ for f in run.sh stop.sh logs.sh scripts/*.sh tests/*.sh; do done if [ -z "$STRAY" ]; then ok; else bad "hard-coded CLI in:$STRAY"; fi +t "run.sh runs both podman preflights, and after RUNTIME is resolved" +if MC=$(grep -n '^runtime_machine_check' run.sh | cut -d: -f1) && \ + RC=$(grep -n '^runtime_rosetta_check' run.sh | cut -d: -f1) && \ + RES=$(grep -n 'runtime_resolve' run.sh | head -1 | cut -d: -f1) && \ + [ -n "$MC" ] && [ -n "$RC" ] && [ "$RES" -lt "$MC" ] && [ "$RES" -lt "$RC" ]; then ok +else bad "machine=${MC:-absent} rosetta=${RC:-absent} resolve=${RES:-absent}"; fi + t "every script that runs containers sources _runtime.sh and resolves before using \$RUNTIME" MISS="" for f in run.sh stop.sh logs.sh scripts/_builder.sh scripts/switch-llm.sh scripts/detect-bedrock.sh; do From 8b0199d07be6ba6d455f45f07c06cdc94a264475 Mon Sep 17 00:00:00 2001 From: Darren Paxton Date: Tue, 29 Sep 2026 11:45:27 -0400 Subject: [PATCH 04/12] fix(runtime): diagnose the machine provider, not just the rosetta key MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rosetta is an applehv feature. podman 6 defaults to libkrun on Apple Silicon, and LibKrunStubber.GetRosetta returns false unconditionally, so `rosetta = true` under libkrun is read and discarded — leaving no trace but the missing binfmt handler. runtime_rosetta_check saw that handler was missing and blamed the config, telling people their file was already correct and the machine just needed restarting. It could never come true, and they recreated the same VM until they gave up. The provider now leads the diagnosis, resolved from the running machine's VMType first (podman machine init --provider does not write containers.conf, so the file describes the NEXT machine, not this one), then the environment, then the config. The remediation splits the two lifetimes that were previously conflated: the provider is fixed at init, so changing it means recreating the VM; the rosetta key is re-read by applehv's StartVM on every start, so on applehv a stop/start really is enough. The old footer promised the latter under every branch. Verified on real hardware both ways: a libkrun machine is told to switch provider and recreate; a rebuilt applehv machine with rosetta = true passes silently, and an amd64 studio image serves /healthz 200 under Rosetta. Co-Authored-By: Claude Opus 5 (1M context) --- docs/getting-started/prerequisites.md | 37 ++++++--- docs/troubleshooting.md | 29 ++++++- scripts/_runtime.sh | 111 ++++++++++++++++++++++++-- tests/test-runtime.sh | 94 ++++++++++++++++++++-- 4 files changed, 247 insertions(+), 24 deletions(-) diff --git a/docs/getting-started/prerequisites.md b/docs/getting-started/prerequisites.md index 87f165e..c850d43 100644 --- a/docs/getting-started/prerequisites.md +++ b/docs/getting-started/prerequisites.md @@ -53,23 +53,40 @@ What matters is that the runtime is reachable and that its `compose` subcommand > podman needs nothing else; the build scripts handle the user-namespace uid mapping for you, so > extension bundles come out owned by you rather than by root. > -> **podman on Apple Silicon** needs two more things before the studio will start, because the studio +> **podman on Apple Silicon** needs three more things before the studio will start, because the studio > image is amd64 on some tags and runs emulated: > -> 1. **Enable Rosetta.** podman does **not** enable it by default, and without it amd64 binaries fall +> 1. **Create the machine with `--provider applehv`.** Rosetta is an applehv feature. podman 6 defaults +> to **libkrun** on Apple Silicon, and libkrun has no Rosetta support at all — upstream's +> `LibKrunStubber.GetRosetta` returns false unconditionally, so a `rosetta = true` under libkrun is +> read and silently discarded, and `podman machine inspect` keeps reporting `Rosetta: false` however +> many times you edit the config. **Unlike the rosetta key, the provider is fixed at +> `podman machine init`** — it is the one setting an existing machine cannot be talked out of, so +> switching means destroying and recreating the VM. +> 2. **Enable Rosetta.** podman does **not** enable it by default, and without it amd64 binaries fall > through to QEMU, which cannot run the studio's .NET runtime. Create -> `~/.config/containers/containers.conf` with `[machine] rosetta = true` before starting the machine -> (the key is read on every `podman machine start`, so an existing machine only needs a stop/start, -> not re-creating). **If that file already exists, edit its `[machine]` section — never append a -> second `[machine]` table, which is a TOML duplicate-key error that stops podman running at all.** -> 2. **Size the machine.** `podman machine init` defaults to 2048 MiB; `run.sh` hard-fails below +> `~/.config/containers/containers.conf` with `[machine] provider = "applehv"` and +> `[machine] rosetta = true` before starting the machine (on applehv the rosetta key is re-read on +> every `podman machine start`, so *that* key only needs a stop/start, not re-creating). **If that +> file already exists, edit its `[machine]` section — never append a second `[machine]` table, which +> is a TOML duplicate-key error that stops podman running at all.** +> 3. **Size the machine.** `podman machine init` defaults to 2048 MiB; `run.sh` hard-fails below > 6144 MiB, because an undersized VM does not stop the stack coming up — it poisons later extension -> builds with an OOM kill that reports itself as `exit status 137`. Use -> `podman machine init --cpus 4 -m 8192 --disk-size 100`. +> builds with an OOM kill that reports itself as `exit status 137`. > -> Verify both before your first run: +> Which makes the whole first-time sequence: > > ```bash +> mkdir -p ~/.config/containers +> printf '[machine]\nprovider = "applehv"\nrosetta = true\n' > ~/.config/containers/containers.conf +> podman machine init --provider applehv --cpus 4 -m 8192 --disk-size 100 +> podman machine start +> ``` +> +> Verify all three before your first run: +> +> ```bash +> podman machine list --format '{{.Name}} {{.VMType}}' # want applehv, NOT libkrun > podman machine ssh 'ls /proc/sys/fs/binfmt_misc/' # want a `rosetta` entry, no `qemu-x86_64` > podman machine list # want MEMORY >= 6144 MiB > ``` diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 30be36d..a2e8885 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -130,10 +130,34 @@ one of them works: - **QEMU user-mode** — the fallback. Cannot run .NET's JIT/reflection; the process takes `SIGABRT` while `MapControllers()` walks controller attributes at startup. -**Rosetta is opt-in, and podman does not enable it for you.** Unless +**First check the provider — under libkrun none of the rest of this applies.** podman 6 defaults to the +`libkrun` provider on Apple Silicon, and libkrun has **no Rosetta support at all**: upstream's +`LibKrunStubber.GetRosetta` returns false unconditionally, so `rosetta = true` is read and discarded +without a warning anywhere. The symptom is identical to a missing key — no `rosetta` binfmt handler, QEMU, +a studio that aborts — but every fix aimed at the config or at restarting the machine is wasted, because +the config was never the problem: + +```bash +podman machine list --format '{{.Name}} {{.VMType}}' # want applehv; libkrun can never have Rosetta +``` + +Unlike the rosetta key, **the provider is fixed at `podman machine init`**. Switching to applehv means +recreating the VM, which destroys its images and volumes: + +```bash +podman machine stop && podman machine rm -f +podman machine init --provider applehv --cpus 4 -m 8192 --disk-size 100 +podman machine start +``` + +Podman Desktop muddies this further: it has a known bug ([#16341](https://github.com/podman-desktop/podman-desktop/issues/16341)) +where it reports Rosetta as enabled on libkrun machines that cannot have it. + +**On applehv, Rosetta is opt-in, and podman does not enable it for you.** Unless `~/.config/containers/containers.conf` sets `[machine] rosetta = true`, the VM is started with no Rosetta share and every amd64 binary falls through to QEMU. The key is read on every `podman machine start`, not -only at `init`. Measured on podman 6.1.2, reading `AppleHypervisor.Vfkit.Rosetta` from the machine's JSON: +only at `init`. Measured on an **applehv** machine, podman 6.1.2, reading `AppleHypervisor.Vfkit.Rosetta` +from the machine's JSON (these numbers do not describe libkrun, which is false in every row): | `containers.conf` | resulting `Vfkit.Rosetta` | | --- | --- | @@ -178,6 +202,7 @@ podman machine ssh 'ls /proc/sys/fs/binfmt_misc/' Corroborate: ```bash +podman machine list --format '{{.VMType}}' # libkrun means Rosetta is impossible, not absent podman machine inspect --format '{{.Rosetta}}' # false when broken podman machine ssh 'mount | grep -i rosetta' # no output when broken podman machine ssh 'systemctl status rosetta-activation.service --no-pager -l' diff --git a/scripts/_runtime.sh b/scripts/_runtime.sh index c1d9e8a..f6a84c9 100644 --- a/scripts/_runtime.sh +++ b/scripts/_runtime.sh @@ -395,6 +395,72 @@ runtime_rosetta_active() { return 1 } +# _runtime_machine_vmtype -> the VMType of the machine this run would talk to, or empty. +# +# The impure half of the provider question, split out for the same reason as runtime_rosetta_active: it +# needs podman and a machine, and the policy that consumes it must be testable without either. Same +# machine-selection rule as runtime_machine_check (running, else default, else first) so both checks +# always talk about the same VM. Anything unreadable is empty, never a guess. +_runtime_machine_vmtype() { + command -v podman >/dev/null 2>&1 || { printf ''; return; } + command -v python3 >/dev/null 2>&1 || { printf ''; return; } + podman machine list --format json 2>/dev/null | python3 -c ' +import json, sys +try: + ms = json.load(sys.stdin) +except Exception: + sys.exit(0) +if not isinstance(ms, list) or not ms: + sys.exit(0) +m = next((x for x in ms if x.get("Running")), None) \ + or next((x for x in ms if x.get("Default")), None) \ + or ms[0] +sys.stdout.write(str(m.get("VMType") or "")) +' 2>/dev/null || printf '' +} + +# _runtime_machine_conf_provider -> the [machine] provider in containers.conf, or empty. +_runtime_machine_conf_provider() { + local f="${CONTAINERS_CONF:-$HOME/.config/containers/containers.conf}" + [ -f "$f" ] || { printf ''; return; } + # Same [machine]-table scan as _runtime_rosetta_conf_state: header to the next [section] or EOF. + local block + block="$(awk '/^[[:space:]]*\[machine\][[:space:]]*$/{f=1;next} /^[[:space:]]*\[/{f=0} f' "$f" 2>/dev/null)" + [ -n "$block" ] || { printf ''; return; } + # _runtime_clean strips quotes BEFORE trimming whitespace, so ` "applehv"` (the leading space `cut` + # leaves behind) would keep its quotes. Trim the left side first and the quote is what it sees. + local v + v="$(printf '%s' "$block" | grep -E '^[[:space:]]*provider[[:space:]]*=' | tail -1 | cut -d= -f2-)" + v="${v#"${v%%[![:space:]]*}"}" + _runtime_clean "$v" +} + +# _runtime_machine_provider -> applehv | libkrun | | empty +# +# Which machine provider is in effect, resolved the way podman resolves it: the environment override +# first, then containers.conf, then — for a machine that already exists — what it was actually created +# as. The last one matters because the provider is fixed at `podman machine init`: editing the config +# afterwards changes what the NEXT machine would be, not this one, so the config alone would cheerfully +# report applehv for a libkrun VM that is running right now. +# +# This exists because Rosetta is an applehv feature. Upstream's LibKrunStubber.GetRosetta returns false +# unconditionally, so `rosetta = true` under libkrun is read, discarded, and leaves no trace anywhere +# except the missing binfmt handler — which is exactly the symptom runtime_rosetta_check already sees +# and, until now, mis-diagnosed as "you forgot to restart the machine". +_runtime_machine_provider() { + local p + # An existing machine answers first, and outranks both config sources: `podman machine init + # --provider applehv` does NOT write containers.conf, so the two disagree the moment anyone passes + # that flag — and it is the running VM, not the file, that this run will talk to. Reading the file + # first would blame libkrun for a healthy applehv machine and advise destroying it, which is the + # same class of wrong answer this function exists to stop. + p="$(_runtime_machine_vmtype)" + # No machine yet: both remaining sources describe what the NEXT one would be, in podman's own order. + [ -n "$p" ] || p="$(_runtime_clean "${CONTAINERS_MACHINE_PROVIDER:-}")" + [ -n "$p" ] || p="$(_runtime_machine_conf_provider)" + printf '%s' "$p" +} + # _runtime_rosetta_conf_state -> absent | no-machine | no-key | disabled | enabled # # What containers.conf currently says about Rosetta, so the remediation can be specific. It has to be, @@ -461,9 +527,37 @@ runtime_rosetta_check() { # podman then refuses to run at all ("Key 'machine' has already been defined") — strictly worse than # the missing Rosetta. And when the key is already correct, the file is not the problem: the machine # simply has not been restarted since it was set. - local conf_path conf_dir fix + local conf_path conf_dir fix provider after conf_path="${CONTAINERS_CONF:-$HOME/.config/containers/containers.conf}" conf_dir="$(dirname "$conf_path")" + provider="$(_runtime_machine_provider)" + + # The provider outranks every conf state, because under anything but applehv the rosetta key is read + # and thrown away: "your config is already correct, just restart" is then advice that cannot ever come + # true, and the person dutifully recreates the machine again and again with the same result. + if [ -n "$provider" ] && [ "$provider" != applehv ]; then + fix="The machine provider is '$provider', and Rosetta is an applehv feature. podman's $provider +backend reports Rosetta as unavailable whatever containers.conf says, so a 'rosetta = true' +there is read and discarded — which is why this looks like a config that ought to work. + +Set BOTH keys under the SINGLE existing [machine] section of +$conf_path +— never add a second [machine] header, TOML rejects a duplicate table and podman then +refuses to run at all: + + [machine] + provider = \"applehv\" + rosetta = true" + after="Then recreate the machine. The PROVIDER, unlike the rosetta key, is written at +'podman machine init' and never revisited, so it cannot be changed in place. +Recreating DESTROYS that VM's images and volumes: + + podman machine stop + podman machine rm -f + podman machine init --provider applehv --cpus 4 -m 8192 --disk-size 100 + podman machine start + ./run.sh" + else case "$(_runtime_rosetta_conf_state)" in absent) fix="No $conf_path yet — create it: @@ -490,12 +584,20 @@ then refuses to start at all: fix="$conf_path ALREADY sets 'rosetta = true' — the file is correct and needs no edit. The key is read at every machine start, so the machine simply has not been restarted since." ;; esac + after="Then restart the machine — it does NOT need to be recreated, and nothing is lost. applehv +re-reads the rosetta key from containers.conf on every start and syncs it into the machine: + + podman machine stop + podman machine start + ./run.sh" + fi cat >&2 < [runtime] -> "|" ( HA="$1"; PL="$2"; PRC="$3" export DUPLO_ENV_FILE=/dev/null RUNTIME="${4:-podman}" STUDIO_PLATFORM="$PL" + # Pin the provider inputs too: left alone, the check reads the developer's real containers.conf and + # shells out to podman, and these cases would then answer differently on an applehv laptop than on + # CI. The provider's own behaviour is asserted through rosmsg below, where it is set explicitly. + export CONTAINERS_CONF=/dev/null; unset CONTAINERS_MACHINE_PROVIDER . ./scripts/_runtime.sh eval "_runtime_host_arch() { printf '%s' '$HA'; }" eval "runtime_rosetta_active() { return $PRC; }" + _runtime_machine_vmtype() { printf ''; } ERR="$(runtime_rosetta_check 2>&1 >/dev/null)"; RC=$? printf '%s|%s' "$RC" "$(printf '%s' "$ERR" | head -1)" ) } @@ -139,17 +144,59 @@ env = [] ')" [ "$R" = no-machine ] && ok || bad "got '$R'" +# Machine provider. Rosetta is an applehv feature: the libkrun provider's GetRosetta returns false +# unconditionally upstream, so `rosetta = true` under libkrun is read and discarded. The provider is +# therefore part of the diagnosis, not a detail. Resolution order mirrors podman's own, and the podman +# call is its own function so the policy is testable without a VM. +prov() { # prov -> the provider word + ( eval "$1" + if [ "$2" != NONE ]; then F="$(mktemp)"; printf '%s' "$2" > "$F"; else F="$(mktemp -u)"; fi + export CONTAINERS_CONF="$F" + . ./scripts/_runtime.sh + if [ "$3" != NONE ]; then eval "_runtime_machine_vmtype() { printf '%s' '$3'; }" + else _runtime_machine_vmtype() { printf ''; }; fi + _runtime_machine_provider ) +} + +t "an existing machine's VMType outranks containers.conf, which only describes the NEXT machine" +# `podman machine init --provider applehv` does not write containers.conf, so the two disagree the +# moment anyone passes the flag — and the VM that is running is the one this run will talk to. Reading +# the config first would blame libkrun for a healthy applehv machine and advise destroying it. +R="$(prov 'unset CONTAINERS_MACHINE_PROVIDER' '[machine] +provider = "libkrun" +' applehv)" +[ "$R" = applehv ] && ok || bad "got '$R'" + +t "with no machine yet, CONTAINERS_MACHINE_PROVIDER outranks containers.conf" +R="$(prov 'export CONTAINERS_MACHINE_PROVIDER=applehv' '[machine] +provider = "libkrun" +' NONE)" +[ "$R" = applehv ] && ok || bad "got '$R'" + +t "with no machine and no environment override, containers.conf says what the next machine will be" +R="$(prov 'unset CONTAINERS_MACHINE_PROVIDER' '[machine] +provider = "libkrun" +' NONE)" +[ "$R" = libkrun ] && ok || bad "got '$R'" + +t "machine provider is empty rather than guessed when nothing can say" +R="$(prov 'unset CONTAINERS_MACHINE_PROVIDER' NONE NONE)" +[ -z "$R" ] && ok || bad "got '$R'" + t "rosetta check fails when an amd64 image would run on an arm64 host without Rosetta" R="$(ros arm64 linux/amd64 1)" case "$R" in 1\|*[Rr]osetta*) ok ;; *) bad "got '$R'" ;; esac -rosmsg() { # rosmsg -> the whole stderr message - ( C="$1" +rosmsg() { # rosmsg [provider] -> the whole stderr message + ( C="$1"; P="${2-}" if [ "$C" != NONE ]; then F="$(mktemp)"; printf '%s' "$C" > "$F"; else F="$(mktemp -u)"; fi export DUPLO_ENV_FILE=/dev/null RUNTIME=podman STUDIO_PLATFORM=linux/amd64 CONTAINERS_CONF="$F" . ./scripts/_runtime.sh _runtime_host_arch() { printf 'arm64'; } runtime_rosetta_active() { return 1; } + # Unset provider (the default) is the "cannot tell" case: the advice must still stand on the config + # alone, which is what every pre-existing case below asserts. + eval "_runtime_machine_provider() { printf '%s' '$P'; }" runtime_rosetta_check 2>&1 >/dev/null ) } @@ -166,12 +213,47 @@ if printf '%s\n' "$M" | grep -qE '>>[[:space:]]*[^|]*containers\.conf'; then bad "tells the user to append a second [machine] table — a TOML duplicate-key error that stops podman" else ok; fi -t "a config that already enables rosetta points at a restart, not at editing the file again" +t "an applehv machine whose config is already right points at a restart, not at recreating" +# applehv's StartVM re-reads cfg.Machine.Rosetta from containers.conf on every start and syncs it into +# the machine config, so the key really does take effect without re-initialising. Only the PROVIDER is +# fixed at init. Telling an applehv user to destroy their VM would cost them images and volumes for a +# change a stop/start already makes. M="$(rosmsg '[machine] rosetta = true -')" -if printf '%s\n' "$M" | grep -qi 'already' && printf '%s\n' "$M" | grep -qi 'podman machine stop'; then ok -else bad "got: $(printf '%s' "$M" | sed -n '9,12p' | tr '\n' ' ')"; fi +' applehv)" +if printf '%s\n' "$M" | grep -qi 'already' && printf '%s\n' "$M" | grep -q 'podman machine start' \ + && ! printf '%s\n' "$M" | grep -q 'podman machine rm'; then ok +else bad "got: $(printf '%s' "$M" | sed -n '9,16p' | tr '\n' ' ')"; fi + +t "a libkrun machine is diagnosed as the provider, not as a missing or unrestarted config" +M="$(rosmsg '[machine] +provider = "libkrun" +rosetta = true +' libkrun)" +if printf '%s\n' "$M" | grep -q 'applehv' && printf '%s\n' "$M" | grep -qi 'libkrun'; then ok +else bad "never names the provider: $(printf '%s' "$M" | sed -n '5,14p' | tr '\n' ' ')"; fi + +t "the libkrun remediation does not send the user back to a config that is already correct" +M="$(rosmsg '[machine] +provider = "libkrun" +rosetta = true +' libkrun)" +if printf '%s\n' "$M" | grep -qi 'rosetta = true.*add\|add.*rosetta = true'; then + bad "tells them to add a key they already have" +else ok; fi + +t "a non-applehv machine is never told a restart will do, because the provider is fixed at init" +# The provider — unlike rosetta — is written at `podman machine init` and never revisited. The old +# footer's "it does NOT need to be recreated, and nothing is lost" is true for applehv and false here, +# and it is the sentence that turns a five-minute fix into an afternoon of recreating the same VM. +M="$(rosmsg '[machine] +provider = "libkrun" +rosetta = true +' libkrun)" +if printf '%s\n' "$M" | grep -qi 'not need to be recreated'; then + bad "promises a restart is enough for a provider that can never honour the key" +elif printf '%s\n' "$M" | grep -q 'podman machine rm'; then ok +else bad "never tells them to recreate: $(printf '%s' "$M" | sed -n '20,28p' | tr '\n' ' ')"; fi t "a missing config gets a create command that cannot clobber anything" M="$(rosmsg NONE)" From 867bf6b34960be5da38d2faab35462dfff21bb04 Mon Sep 17 00:00:00 2001 From: Darren Paxton Date: Wed, 30 Sep 2026 11:48:12 -0400 Subject: [PATCH 05/12] fix(runtime): create .env before resolving the runtime, not after MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit runtime_resolve reads RUNTIME from .env and then memoises its answer in _RUNTIME_RESOLVED for the rest of the run. It ran at run.sh:129, but .env was not created until run.sh:162 — so on a first run there was nothing to read, auto-detection picked docker on presence, and that choice was locked in for both podman preflights, `compose pull` and `compose up`. The effect was the feature's headline promise failing on exactly the first run, on any machine with Docker installed alongside podman: RUNTIME=podman in .env was ignored, and so was shipping it in .env.example, since the copy happened after resolution. Creating .env needs neither python3 nor $RUNTIME, so it moves above the prerequisite block with a comment recording why the order matters. Reported in review of #18. Co-Authored-By: Claude Opus 5 (1M context) --- run.sh | 9 ++++++++- tests/test-runtime.sh | 9 +++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/run.sh b/run.sh index 6c6959d..f9bd35a 100755 --- a/run.sh +++ b/run.sh @@ -102,6 +102,14 @@ while [ $# -gt 0 ]; do shift done +# ── .env must exist before anything reads it ────────────────────────────────── +# Ahead of the prerequisite checks deliberately: runtime_resolve below reads RUNTIME from .env and then +# memoises its answer for the whole run. Creating .env after that point meant a first run on a box with +# both CLIs installed auto-detected docker and ignored RUNTIME=podman — set in .env or shipped in +# .env.example — for both podman preflights, `compose pull` and `compose up`. Needs neither python3 nor +# $RUNTIME, so it is safe this early. +[ -f "$ENV" ] || { [ -f .env.example ] && cp .env.example "$ENV" || touch "$ENV"; } + # ── prerequisites ──────────────────────────────────────────────────────────── # Checked here, after flag parsing (so --help still works on a bare machine) and before the first line # that needs either one — setenv() below is already python3. Both are hard requirements, so report every @@ -159,7 +167,6 @@ runtime_machine_check || exit 1 runtime_rosetta_check || exit 1 # ── .env helpers (line-based; safe for tokens/keys with special chars) ──────── -[ -f "$ENV" ] || { [ -f .env.example ] && cp .env.example "$ENV" || touch "$ENV"; } getenv() { grep -E "^$1=" "$ENV" 2>/dev/null | head -1 | cut -d= -f2- || true; } setenv() { python3 - "$ENV" "$1" "${2-}" <<'PY' diff --git a/tests/test-runtime.sh b/tests/test-runtime.sh index 8cfd6e3..80a4412 100755 --- a/tests/test-runtime.sh +++ b/tests/test-runtime.sh @@ -347,6 +347,15 @@ if sed 's/[[:space:]]*#.*$//' docker-compose.yml | grep -qE '\$\{[^}]*\$\{'; the t ".env.example documents RUNTIME and says runc is not a valid value" if grep -q '^#RUNTIME=' .env.example && grep -qi 'runc' .env.example; then ok; else bad ".env.example"; fi +t "run.sh creates .env before it resolves the runtime" +# runtime_resolve reads RUNTIME from .env and memoises itself, so if .env is created after it runs, a +# first run on a box with both CLIs installed auto-detects docker and ignores RUNTIME=podman entirely -- +# for both preflights, `compose pull` and `compose up`. +ENVL="$(grep -n 'cp .env.example' run.sh | head -1 | cut -d: -f1)" +RESL="$(grep -nE '^runtime_resolve' run.sh | head -1 | cut -d: -f1)" +if [ -n "$ENVL" ] && [ -n "$RESL" ] && [ "$ENVL" -lt "$RESL" ]; then ok +else bad ".env created at line ${ENVL:-?} but runtime resolved at ${RESL:-?} -- RUNTIME in .env is ignored on a first run"; fi + t "bash -n on every shell script in the kit" BADF="" for f in run.sh stop.sh logs.sh scripts/*.sh tests/*.sh; do bash -n "$f" 2>/dev/null || BADF="$BADF $f"; done From 26656b1bb82aa4a7eabbc81ca6bcfaf02882de7a Mon Sep 17 00:00:00 2001 From: Darren Paxton Date: Wed, 30 Sep 2026 11:48:32 -0400 Subject: [PATCH 06/12] fix(runtime): match a [machine] header that carries a trailing comment MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The conf scan anchored on `^[[:space:]]*\[machine\][[:space:]]*$`, so a valid TOML header with a trailing comment did not match: [containers] [machine] # my settings rosetta = true That file reported state `no-machine`, whose remediation tells the user to ADD a [machine] section — producing the `Key 'machine' has already been defined` breakage the function's own header comment exists to prevent, and doing it to a user whose config was already correct. Strictly worse than the missing Rosetta it set out to diagnose. Allows an optional comment after the header in all three places that scan for it (two awk block extractions and the grep that distinguishes no-key from no-machine), plus a fixture case in the conf helper. Verified against a clean-header control, which still reports `enabled`, so the trailing comment was the only variable. Reported in review of #18. Co-Authored-By: Claude Opus 5 (1M context) --- scripts/_runtime.sh | 6 +++--- tests/test-runtime.sh | 10 ++++++++++ 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/scripts/_runtime.sh b/scripts/_runtime.sh index f6a84c9..16256e2 100644 --- a/scripts/_runtime.sh +++ b/scripts/_runtime.sh @@ -425,7 +425,7 @@ _runtime_machine_conf_provider() { [ -f "$f" ] || { printf ''; return; } # Same [machine]-table scan as _runtime_rosetta_conf_state: header to the next [section] or EOF. local block - block="$(awk '/^[[:space:]]*\[machine\][[:space:]]*$/{f=1;next} /^[[:space:]]*\[/{f=0} f' "$f" 2>/dev/null)" + block="$(awk '/^[[:space:]]*\[machine\][[:space:]]*(#.*)?$/{f=1;next} /^[[:space:]]*\[/{f=0} f' "$f" 2>/dev/null)" [ -n "$block" ] || { printf ''; return; } # _runtime_clean strips quotes BEFORE trimming whitespace, so ` "applehv"` (the leading space `cut` # leaves behind) would keep its quotes. Trim the left side first and the quote is what it sees. @@ -481,8 +481,8 @@ _runtime_rosetta_conf_state() { [ -f "$f" ] || { printf 'absent'; return; } # The [machine] table runs from its header to the next [section] header or EOF. local block - block="$(awk '/^[[:space:]]*\[machine\][[:space:]]*$/{f=1;next} /^[[:space:]]*\[/{f=0} f' "$f" 2>/dev/null)" - [ -n "$block" ] || { grep -qE '^[[:space:]]*\[machine\][[:space:]]*$' "$f" 2>/dev/null \ + block="$(awk '/^[[:space:]]*\[machine\][[:space:]]*(#.*)?$/{f=1;next} /^[[:space:]]*\[/{f=0} f' "$f" 2>/dev/null)" + [ -n "$block" ] || { grep -qE '^[[:space:]]*\[machine\][[:space:]]*(#.*)?$' "$f" 2>/dev/null \ && { printf 'no-key'; return; }; printf 'no-machine'; return; } case "$(printf '%s' "$block" | grep -E '^[[:space:]]*rosetta[[:space:]]*=' | tail -1)" in *true*) printf 'enabled' ;; diff --git a/tests/test-runtime.sh b/tests/test-runtime.sh index 80a4412..e13b92c 100755 --- a/tests/test-runtime.sh +++ b/tests/test-runtime.sh @@ -137,6 +137,16 @@ cpus = 4 ')" [ "$R" = no-key ] && ok || bad "got '$R'" +t "conf state finds [machine] when the header carries a trailing comment" +# `[machine] # my settings` is valid TOML. Anchoring the scan on end-of-line after the header hid the +# section, and the resulting no-machine advice tells the user to ADD [machine] — a duplicate-key error +# that stops podman outright, on a user whose config was already correct. +R="$(conf '[containers] +[machine] # my settings +rosetta = true +')" +[ "$R" = enabled ] && ok || bad "got '$R' (a commented header must not read as a missing section)" + t "conf state reports no-machine when the file exists with no [machine] section" R="$(conf '[containers] [engine] From 09b6a192bd5fd11d79739fadf15ad569f0e33352 Mon Sep 17 00:00:00 2001 From: Darren Paxton Date: Wed, 30 Sep 2026 11:48:53 -0400 Subject: [PATCH 07/12] docs(runtime): name brew only for podman-compose, not apt or dnf Linux is not supported right now, so leading with `apt install podman-compose` sends the large majority of users to a command that does not exist on their machine. Both places this PR added it (the missing-compose diagnostic in run.sh and the Apple Silicon setup in prerequisites.md) now name brew only. The pre-existing python3 apt/dnf hint at run.sh:123 is left alone: it predates this branch, so the accompanying test is scoped to the podman-compose guidance rather than every install hint in the kit. Reported in review of #18. Co-Authored-By: Claude Opus 5 (1M context) --- docs/getting-started/prerequisites.md | 2 +- run.sh | 2 +- tests/test-runtime.sh | 7 +++++++ 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/getting-started/prerequisites.md b/docs/getting-started/prerequisites.md index c850d43..fc0a2bf 100644 --- a/docs/getting-started/prerequisites.md +++ b/docs/getting-started/prerequisites.md @@ -49,7 +49,7 @@ What matters is that the runtime is reachable and that its `compose` subcommand ``` > **podman:** install the compose provider too — podman shells out to one rather than implementing -> compose itself (`sudo apt-get install podman-compose`, or `brew install podman-compose`). Rootless +> compose itself (`brew install podman-compose`). Rootless > podman needs nothing else; the build scripts handle the user-namespace uid mapping for you, so > extension bundles come out owned by you rather than by root. > diff --git a/run.sh b/run.sh index f9bd35a..4b4bb59 100755 --- a/run.sh +++ b/run.sh @@ -143,7 +143,7 @@ if [ "$RUNTIME_OK" = 1 ] && ! "$RUNTIME" compose version >/dev/null 2>&1; then • $RUNTIME compose — '$RUNTIME compose version' failed. A v2-style compose subcommand is required; the standalone docker-compose v1 binary is not enough. docker — install the Compose v2 plugin, or upgrade Docker Desktop. - podman — install the provider it delegates to: apt install podman-compose (or dnf/brew)." + podman — install the provider it delegates to: brew install podman-compose." fi if [ -n "$MISSING" ]; then echo "This kit needs a couple of things that aren't here yet:$MISSING" >&2 diff --git a/tests/test-runtime.sh b/tests/test-runtime.sh index e13b92c..38e0b90 100755 --- a/tests/test-runtime.sh +++ b/tests/test-runtime.sh @@ -366,6 +366,13 @@ RESL="$(grep -nE '^runtime_resolve' run.sh | head -1 | cut -d: -f1)" if [ -n "$ENVL" ] && [ -n "$RESL" ] && [ "$ENVL" -lt "$RESL" ]; then ok else bad ".env created at line ${ENVL:-?} but runtime resolved at ${RESL:-?} -- RUNTIME in .env is ignored on a first run"; fi +t "runtime install guidance names brew only, not apt or dnf" +# Linux is not supported right now, so apt/dnf guidance sends the large majority of users to a command +# that does not exist on their machine. +# Scoped to the podman-compose guidance this PR added; the pre-existing python3 hint is not ours. +HITS="$(grep -n 'podman-compose' run.sh docs/getting-started/prerequisites.md 2>/dev/null | grep -E 'apt|dnf')" +if [ -z "$HITS" ]; then ok; else bad "non-brew install guidance: $HITS"; fi + t "bash -n on every shell script in the kit" BADF="" for f in run.sh stop.sh logs.sh scripts/*.sh tests/*.sh; do bash -n "$f" 2>/dev/null || BADF="$BADF $f"; done From a470ade868542ddc297c28fc52ebfff6f096eab9 Mon Sep 17 00:00:00 2001 From: Darren Paxton Date: Wed, 30 Sep 2026 13:02:40 -0400 Subject: [PATCH 08/12] fix(runtime): stop telling people to install a runtime they already have MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The error:runtime branch this PR introduced used one message for three different situations, and got two of them wrong: ERROR: no container runtime is available and the local toolchain is incomplete. Install docker or podman (with a 'compose' subcommand) ... Anyone who had simply not started Docker Desktop was told to install docker. That is the most common way to reach this branch — Docker Desktop does not auto-start unless you enable it — and the advice does not even mention starting anything. Worse on a box with both runtimes: detection is presence-based and docker-first, so a stopped docker beside a healthy podman resolves to docker, refuses to build, and recommends installing the podman that is already running. Hit in practice on a libkrun podman box. Detection stays presence-only — a daemon probe on every run is not worth the second — so the fix is to make the failure self-diagnosing rather than to change what gets picked: runtime_alternatives the supported runtimes OTHER than the resolved one that actually answer, over a _runtime_answers seam so the policy is testable with no runtime installed. builder_runtime_message pure formatter, four cases. Nothing on PATH -> install (unchanged, it was right). Present but dead -> start it, with the per-runtime command. Another one answers -> use it, with the exact RUNTIME= invocation. Explicitly requested -> same, but never phrased as though they had not chosen it. run.sh gets the same treatment: its not-ready note now names a working alternative, and it announces the runtime it picked next to "Pulling images" rather than leaving a presence-based choice invisible until it breaks — the non-blocking item from the review. Tests: 55 (was 45). runtime_alternatives is mutation-tested in both directions: including the resolved runtime, or skipping the reachability probe, each makes specific tests fail. Reported in review of #18. Co-Authored-By: Claude Opus 5 (1M context) --- run.sh | 18 +++++++++- scripts/_builder.sh | 63 +++++++++++++++++++++++++++++++--- scripts/_runtime.sh | 28 ++++++++++++++++ tests/test-runtime.sh | 78 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 182 insertions(+), 5 deletions(-) diff --git a/run.sh b/run.sh index 4b4bb59..1d58450 100755 --- a/run.sh +++ b/run.sh @@ -150,7 +150,20 @@ if [ -n "$MISSING" ]; then RUNTIME_OK=0 fi [ "$RUNTIME_OK" = 1 ] || exit 1 -"$RUNTIME" info >/dev/null 2>&1 || echo "Note: $RUNTIME doesn't look ready (daemon not running, or no connection) — sort that out before this gets to 'Pulling images'." >&2 +# If the resolved runtime is not answering, say so — and if another supported one IS, name it. Detection +# is presence-based, so a stopped Docker Desktop alongside a healthy podman resolves to docker and then +# fails at 'Pulling images' with nothing pointing at the way out. +if ! "$RUNTIME" info >/dev/null 2>&1; then + echo "Note: $RUNTIME doesn't look ready (daemon not running, or no connection) — sort that out before this gets to 'Pulling images'." >&2 + _alt="$(runtime_alternatives)" + if [ -n "$_alt" ]; then + for _a in $_alt; do + echo " $_a is installed and responding. To use it instead: RUNTIME=$_a ./run.sh" >&2 + echo " (or make it this checkout's default: echo 'RUNTIME=$_a' >> .env)" >&2 + done + fi + unset _alt _a +fi # Sized-VM check. Separate from the MISSING one-pass above because it needs RUNTIME already resolved, # and it is a hard failure by design: an undersized podman machine does not stop the stack from coming @@ -1028,6 +1041,9 @@ for v in STUDIO_TAG UI_TAG; do done # ── start the stack ─────────────────────────────────────────────────────────── +# Name the runtime here, not only when something breaks: auto-detect is presence-based and docker-first, +# so on a box with both installed the choice is otherwise invisible until it goes wrong. +echo "==> Using $(runtime_label)" echo "==> Pulling images…"; "$RUNTIME" compose pull echo "==> Starting…"; "$RUNTIME" compose up -d diff --git a/scripts/_builder.sh b/scripts/_builder.sh index ebaa945..55a4dfc 100644 --- a/scripts/_builder.sh +++ b/scripts/_builder.sh @@ -134,6 +134,63 @@ builder_probe_runtime() { if "$RUNTIME" version --format '{{json .Server}}' >/dev/null 2>&1; then echo 1; else echo 0; fi } +# builder_runtime_message +# +# Formats the "this build cannot be containerized" failure. Pure — it probes nothing — so every branch +# below is exercised in the tests on any machine. +# +# One message used to cover three situations and got two of them wrong. It said "Install docker or +# podman" to anyone who had merely not started Docker Desktop, which is the most common way to arrive +# here, and to anyone running a perfectly healthy podman that detection had passed over. What is +# actually true differs per case, so the text does too: +# +# nothing on PATH -> installing really is the fix +# present but not answering -> start it +# ...and another one answers -> use that one, with the command that does it +builder_runtime_message() { + local rt="${1-}" alts="${2-}" missing="${3-}" requested="${4-0}" a + + if [ -z "$rt" ]; then + echo "ERROR: no container runtime found on PATH, and the local toolchain is incomplete." >&2 + echo " Missing: ${missing:-(none)}" >&2 + echo " Install docker or podman (each needs a v2-style 'compose' subcommand) — that is the" >&2 + echo " only prerequisite — or install the toolchain yourself and re-run with --native." >&2 + return + fi + + if [ "$requested" = 1 ]; then + echo "ERROR: '$rt' was requested (RUNTIME) but is not responding, so this build cannot be" >&2 + echo " containerized." >&2 + else + echo "ERROR: '$rt' was auto-detected but is not responding, so this build cannot be containerized." >&2 + echo " Detection tests whether a CLI is on PATH, not whether it is running." >&2 + fi + + if [ -n "$alts" ]; then + echo "" >&2 + # shellcheck disable=SC2086 # deliberate word-split: alts is a space-separated list + for a in $alts; do + echo " $a is installed and responding. Use it for this build:" >&2 + echo "" >&2 + echo " RUNTIME=$a ./scripts/build-extension.sh " >&2 + echo "" >&2 + echo " ...or make it this checkout's default: echo 'RUNTIME=$a' >> .env" >&2 + done + echo " ...or start $rt and re-run." >&2 + else + echo "" >&2 + case "$rt" in + docker) echo " Start it and re-run — on macOS, launch Docker Desktop and wait for it to" >&2 + echo " report Running." >&2 ;; + podman) echo " Start it and re-run: podman machine start" >&2 ;; + *) echo " Start it and re-run." >&2 ;; + esac + fi + + echo "" >&2 + echo " Or build with a toolchain you installed yourself: --native (missing now: ${missing:-none})" >&2 +} + builder_missing_tools() { local t out="" for t in "${_BUILDER_TOOLS[@]}"; do @@ -399,10 +456,8 @@ builder_dispatch() { echo " build/Dockerfile.builder in this repo." >&2 exit 1 ;; error:runtime) - echo "ERROR: no container runtime is available and the local toolchain is incomplete." >&2 - echo " Missing: $(builder_missing_tools)" >&2 - echo " Install docker or podman (with a 'compose' subcommand) — that is the" >&2 - echo " only prerequisite — or install the toolchain yourself and re-run with --native." >&2 + builder_runtime_message "${RUNTIME:-}" "$(runtime_alternatives)" "$(builder_missing_tools)" \ + "$([ -n "$(runtime_requested)" ] && echo 1 || echo 0)" exit 1 ;; container) ;; # error:badarg, or a mode this function hasn't been taught. The only way to reach it in practice is diff --git a/scripts/_runtime.sh b/scripts/_runtime.sh index 16256e2..8597c90 100644 --- a/scripts/_runtime.sh +++ b/scripts/_runtime.sh @@ -88,6 +88,34 @@ runtime_detect() { # The distinction matters to the build path: "you asked for X and X is unusable" is a hard error, # because silently building with something else is not what was asked for — whereas "nothing found" # can legitimately fall back to a native toolchain build. +# _runtime_answers -> 0 when that CLI is present AND something answers on the other end. +# +# Split out from runtime_alternatives for the reason runtime_rosetta_active is split from the Rosetta +# policy: the probe needs a real runtime, and the policy consuming it has to be testable without one. +# Same `version --format '{{json .Server}}'` form as builder_probe_runtime, and for the same reason — +# rendering any part of the SERVER block requires contacting the server, while the whole-object form +# still renders on a CLI whose server block is shaped differently (finch) instead of dying in the +# template engine and reading as "unreachable". +_runtime_answers() { + command -v "$1" >/dev/null 2>&1 || return 1 + "$1" version --format '{{json .Server}}' >/dev/null 2>&1 +} + +# runtime_alternatives -> the supported runtimes OTHER than the resolved one that actually answer. +# +# Exists so a failure can name a way out instead of reciting prerequisites. Detection is presence-based +# and docker-first (deliberately: a daemon check on every run is not worth the second), so a box with +# Docker Desktop installed-but-stopped alongside a healthy podman resolves to docker and then cannot +# build. The honest advice there is "use podman", which requires knowing podman answers. +runtime_alternatives() { + local r out="" + for r in "${_RUNTIME_SUPPORTED[@]}"; do + [ "$r" = "${RUNTIME:-}" ] && continue + _runtime_answers "$r" && out="$out $r" + done + printf '%s' "${out# }" +} + runtime_requested() { _runtime_clean "${RUNTIME:-$(_runtime_envv RUNTIME)}"; } # runtime_resolve -> sets and exports RUNTIME; returns 1 with a message on stderr if it cannot. diff --git a/tests/test-runtime.sh b/tests/test-runtime.sh index 38e0b90..e47cfad 100755 --- a/tests/test-runtime.sh +++ b/tests/test-runtime.sh @@ -307,6 +307,84 @@ else fi echo +echo +echo "runtime failure messages:" + +# runtime_alternatives asks, of the runtimes we did NOT pick, which ones actually answer. The reachability +# probe is its own function so the policy is testable with no runtime installed at all — the same seam +# runtime_rosetta_active provides for the Rosetta check. +alts() { # alts -> runtime_alternatives output + ( export DUPLO_ENV_FILE=/dev/null RUNTIME="$1"; ANS=" $2 " + . ./scripts/_runtime.sh + eval '_runtime_answers() { case "$ANS" in *" $1 "*) return 0 ;; *) return 1 ;; esac; }' + runtime_alternatives ) +} + +t "alternatives never include the runtime we already resolved to" +[ "$(alts docker 'docker podman')" = podman ] && ok || bad "got '$(alts docker 'docker podman')'" + +t "alternatives list only runtimes that actually answer" +[ "$(alts docker '')" = "" ] && ok || bad "got '$(alts docker '')' for a box where nothing answers" + +t "alternatives are empty when the only other runtime is installed but dead" +[ "$(alts podman 'podman')" = "" ] && ok || bad "got '$(alts podman 'podman')'" + +# The message itself is pure: given what was resolved, what else answers, and what the toolchain lacks, +# it only formats. No probing, so every case below is exercised on any machine. +msg() { # msg -> the whole message + ( export DUPLO_ENV_FILE=/dev/null + . ./scripts/_builder.sh + builder_runtime_message "$1" "$2" "$3" "$4" 2>&1 ) +} + +t "case A: an auto-detected but dead runtime names the working alternative and how to use it" +M="$(msg docker podman dotnet 0)" +if grep -q 'podman' <<<"$M" && grep -q 'RUNTIME=podman' <<<"$M" && grep -qiE 'not responding' <<<"$M" +then ok; else bad "$M"; fi + +t "case A: it never tells you to install the runtime you already have" +M="$(msg docker podman dotnet 0)" +if grep -qiE 'install docker|install docker or podman' <<<"$M"; then + bad "tells the user to install docker, which is installed — the whole bug" +else ok; fi + +t "case A-prime: an EXPLICITLY requested runtime is not described as auto-detected" +M="$(msg docker podman dotnet 1)" +if grep -qi 'auto-detected' <<<"$M"; then bad "implies they did not choose it: $M" +elif grep -qiE 'requested|asked for' <<<"$M" && grep -q 'RUNTIME=podman' <<<"$M"; then ok +else bad "$M"; fi + +t "case B: a dead runtime with no alternative says to START it, not to install it" +M="$(msg docker '' dotnet 0)" +if grep -qiE 'install docker' <<<"$M"; then bad "says install, but docker is present: $M" +elif grep -qiE 'start' <<<"$M" && grep -qiE 'not responding|installed but' <<<"$M"; then ok +else bad "$M"; fi + +t "case B: the --native escape hatch is offered and names what is missing" +M="$(msg docker '' dotnet 0)" +if grep -q -- '--native' <<<"$M" && grep -q 'dotnet' <<<"$M"; then ok; else bad "$M"; fi + +t "case C: with nothing on PATH at all, installing IS the right advice" +M="$(msg '' '' dotnet 0)" +if grep -qiE 'install' <<<"$M" && grep -qiE 'on PATH|found' <<<"$M"; then ok; else bad "$M"; fi + +t "case C is distinguishable from case B in the first line" +if [ "$(msg '' '' dotnet 0 | head -1)" != "$(msg docker '' dotnet 0 | head -1)" ]; then ok +else bad "'nothing installed' and 'installed but stopped' open identically"; fi + +t "run.sh's not-ready warning names a working alternative when there is one" +if grep -A6 'doesn.t look ready\|not responding' run.sh | grep -q 'runtime_alternatives'; then ok +else bad "run.sh warns the runtime is dead without mentioning the healthy one sitting next to it"; fi + +t "run.sh says which runtime it picked on the happy path, not only on failure" +# Presence-based auto-detect means the choice is invisible until something breaks; one line fixes that. +if grep -B4 'Pulling images' run.sh | grep -qE 'runtime_label|using .*RUNTIME|\$RUNTIME"?\)'; then ok +else bad "no runtime announced next to 'Pulling images'"; fi + +t "builder_dispatch's error:runtime branch delegates to the message function" +if grep -A4 'error:runtime)' scripts/_builder.sh | grep -q 'builder_runtime_message'; then ok +else bad "the branch still inlines its own text, so none of the above applies to a real build"; fi + echo "no stray hard-coded docker calls:" # The scripts that drive containers. Anything invoking the CLI must go through $RUNTIME so that the From 7b4060aca5e6c9e56dcbf20134e07317e399f802 Mon Sep 17 00:00:00 2001 From: Darren Paxton Date: Wed, 30 Sep 2026 13:18:52 -0400 Subject: [PATCH 09/12] docs(runtime): explain the cross-runtime build warning instead of leaving it to be rediscovered MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Running the stack under one runtime and building under another produces a warning that looks like a broken build and is not: WARNING: no running duplo-ai-studio in this compose project … will try the published host port instead: http://host.docker.internal:60031 Compose scopes services per project per runtime, so a build under docker cannot see a studio started under podman. The fallback reaches the studio through its published host port — the same one the browser uses — so the SDK fetch succeeds and the bundle is correct. Easy to hit, because detection is presence-based and docker-first: `RUNTIME=podman ./run.sh` for the stack, then a build with no RUNTIME set, and the two are on different runtimes without either command being wrong. The part worth writing down is the confirmation step, because its result reads as a failure: the SDK feed is authenticated, so probing it unauthenticated from inside the builder returns **401 — which proves the route works**. Anyone checking this without being told will read the 401 as the fault and go looking for a token problem that does not exist. Also makes the neighbouring entry's `docker compose ps` runtime-agnostic, since it sits two paragraphs from an entry about runtimes not matching. Co-Authored-By: Claude Opus 5 (1M context) --- docs/troubleshooting.md | 41 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 40 insertions(+), 1 deletion(-) diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index a2e8885..9460913 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -569,7 +569,7 @@ up and reachable at the resolved base URL before a build can start. With `DUPLO_ - **Local** — confirm the stack is running and the port matches `.env`: ```bash - docker compose ps + ./logs.sh --no-follow duplo-ai-studio # or: compose ps grep STUDIO_PORT .env ``` @@ -588,6 +588,45 @@ A version string like `{"version":"1.0.6"}` means the build will get past this s **authenticated** — without the header you get a 401, which is not the same problem as the studio being down. For that, `curl -fsS http://localhost:60031/healthz` answers anonymously. +### The build warns it cannot see the studio, then works anyway + +**Symptom** — `./scripts/build-extension.sh` prints a warning, then the build carries on and succeeds: + +``` +WARNING: no running duplo-ai-studio in this compose project (docker compose scopes services + per project) … will try the published host port instead: + http://host.docker.internal:60031 +``` + +**Why** — the build and the stack are on **different container runtimes**. Compose scopes services per +project *per runtime*, so a build running under docker cannot see a studio that was started under podman, +or the reverse. It is easy to end up here because detection is presence-based and docker-first: you may +have started the stack with `RUNTIME=podman ./run.sh` while the build, run without that variable, +auto-detects docker. + +**This is usually benign.** The fallback URL reaches the studio through its *published host port* — the +same port your browser uses — so the SDK fetch still succeeds and the bundle builds correctly. Treat it as +noise unless the build actually fails on the SDK step, which is the previous entry. + +**Confirm the fallback route is alive** — from inside a container on the build's runtime: + +```bash +RT=docker # the runtime the BUILD uses, not the stack's +IMG=$(grep ^BUILDER_IMAGE= .env | cut -d= -f2-) +PORT=$(grep ^STUDIO_PORT= .env | cut -d= -f2-); PORT=${PORT:-60021} +$RT run --rm --add-host host.docker.internal:host-gateway "$IMG" \ + curl -sS -o /dev/null -w '%{http_code}\n' \ + "http://host.docker.internal:$PORT/v1/aiservicedesk/extensions/sdk-version" +``` + +**`401` means the route is fine.** The SDK feed is authenticated, so an unauthenticated request from inside +the container is *supposed* to be rejected — reaching a 401 proves the network path works. A connection +error or `000` is the real failure. Do not read the 401 as the problem. + +**Silence it** — put both on the same runtime, e.g. `echo 'RUNTIME=podman' >> .env`, so the build and the +stack share a compose project. The builder image is cached per runtime, so the first build after switching +pays a one-time pull. + ### The extension's page never loads in the portal The portal loads extension remotes with Native Federation and reads a `remoteEntry.json`. Check, in From b442ceae62d718040aeaf03f20c1745f830f90ab Mon Sep 17 00:00:00 2001 From: Darren Paxton Date: Wed, 30 Sep 2026 14:30:33 -0400 Subject: [PATCH 10/12] docs(runtime): podman needs a compose provider, not specifically podman-compose MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The earlier fix for the apt/dnf guidance overshot: it replaced one wrong package manager with one package, and `podman-compose` is not the only provider podman will delegate to. A `docker-compose` binary satisfies `podman compose` just as well, and Podman Desktop installs one during its own setup. Verified on a machine with no docker installed at all: `podman-compose` absent, `/usr/local/bin/docker-compose` present, and `podman compose version` answering — so the previous text told that user to install a second provider for a prerequisite they had already met, under a different name. Both places now describe the requirement rather than a package, and point at the check `run.sh` actually runs (`podman compose version`), so the advice stays true whichever provider is installed. The accompanying test asserts that anywhere podman-compose is named, an alternative route is named too — the failure mode being fixed is naming exactly one. Co-Authored-By: Claude Opus 5 (1M context) --- docs/getting-started/prerequisites.md | 6 ++++-- run.sh | 3 ++- tests/test-runtime.sh | 12 ++++++++++++ 3 files changed, 18 insertions(+), 3 deletions(-) diff --git a/docs/getting-started/prerequisites.md b/docs/getting-started/prerequisites.md index fc0a2bf..d4d1bc9 100644 --- a/docs/getting-started/prerequisites.md +++ b/docs/getting-started/prerequisites.md @@ -48,8 +48,10 @@ What matters is that the runtime is reachable and that its `compose` subcommand bash -c 'source scripts/_runtime.sh; echo "using: $(runtime_detect)"' ``` -> **podman:** install the compose provider too — podman shells out to one rather than implementing -> compose itself (`brew install podman-compose`). Rootless +> **podman:** it needs a compose provider, because podman shells out to one rather than implementing +> compose itself. Podman Desktop installs one during setup (a `docker-compose` binary, which podman +> delegates to), or `brew install podman-compose`. Either satisfies the `podman compose version` check +> `run.sh` runs — so if that already answers, there is nothing to install. Rootless > podman needs nothing else; the build scripts handle the user-namespace uid mapping for you, so > extension bundles come out owned by you rather than by root. > diff --git a/run.sh b/run.sh index 1d58450..710a813 100755 --- a/run.sh +++ b/run.sh @@ -143,7 +143,8 @@ if [ "$RUNTIME_OK" = 1 ] && ! "$RUNTIME" compose version >/dev/null 2>&1; then • $RUNTIME compose — '$RUNTIME compose version' failed. A v2-style compose subcommand is required; the standalone docker-compose v1 binary is not enough. docker — install the Compose v2 plugin, or upgrade Docker Desktop. - podman — install the provider it delegates to: brew install podman-compose." + podman — it shells out to a compose provider rather than implementing compose itself. + Podman Desktop can install one for you, or: brew install podman-compose" fi if [ -n "$MISSING" ]; then echo "This kit needs a couple of things that aren't here yet:$MISSING" >&2 diff --git a/tests/test-runtime.sh b/tests/test-runtime.sh index e47cfad..f87b482 100755 --- a/tests/test-runtime.sh +++ b/tests/test-runtime.sh @@ -444,6 +444,18 @@ RESL="$(grep -nE '^runtime_resolve' run.sh | head -1 | cut -d: -f1)" if [ -n "$ENVL" ] && [ -n "$RESL" ] && [ "$ENVL" -lt "$RESL" ]; then ok else bad ".env created at line ${ENVL:-?} but runtime resolved at ${RESL:-?} -- RUNTIME in .env is ignored on a first run"; fi +t "podman compose guidance names more than one way to get a provider" +# podman needs *a* compose provider, not specifically podman-compose: a docker-compose binary satisfies +# `podman compose` just as well, and Podman Desktop installs one. Naming only the brew package sends +# people to install a second provider they may already have under another name. +MISS="" +for f in run.sh docs/getting-started/prerequisites.md; do + if grep -q 'podman-compose' "$f" 2>/dev/null; then + grep -qi 'podman desktop' "$f" 2>/dev/null || MISS="$MISS $f" + fi +done +if [ -z "$MISS" ]; then ok; else bad "names only podman-compose, no alternative route:$MISS"; fi + t "runtime install guidance names brew only, not apt or dnf" # Linux is not supported right now, so apt/dnf guidance sends the large majority of users to a command # that does not exist on their machine. From d9266abcf5bb5b2a5bc0b5d8efa6a3c87618f4d9 Mon Sep 17 00:00:00 2001 From: Darren Paxton Date: Wed, 30 Sep 2026 15:15:09 -0400 Subject: [PATCH 11/12] fix(runtime): name a working runtime in the compose preflight too MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The alternative-naming added for the build failure and the readiness warning skipped the one preflight most likely to hit it. Reported from real use: `.env` pinned RUNTIME=podman, podman's socket had broken, docker was up with Compose v2 — and run.sh printed install advice for both runtimes without mentioning that docker would have worked immediately. Every line of that advice was correct and none of it was relevant, which is the same failure the error:runtime message had: reciting prerequisites at someone who already meets them. runtime_alternatives_compose is deliberately narrower than runtime_alternatives. A runtime can be reachable and still have no compose subcommand, and offering that one as the way out of a compose failure sends the person in a circle — so both conditions are required, each behind its own probe (_runtime_answers, _runtime_has_compose) so the policy is testable with neither runtime installed. The message also names the pin as the likely cause, because that is what it was: auto-detection would have picked docker, and a RUNTIME in .env silently outranks it. Tests: 60 (was 56). Both conditions are mutation-tested — dropping the compose requirement or the resolved-runtime exclusion each makes a specific test fail. Verified live on the machine that reported it: with RUNTIME=podman forced and podman broken, it names docker. Co-Authored-By: Claude Opus 5 (1M context) --- run.sh | 11 +++++++++++ scripts/_runtime.sh | 23 +++++++++++++++++++++++ tests/test-runtime.sh | 26 ++++++++++++++++++++++++++ 3 files changed, 60 insertions(+) diff --git a/run.sh b/run.sh index 710a813..aed10ca 100755 --- a/run.sh +++ b/run.sh @@ -145,6 +145,17 @@ if [ "$RUNTIME_OK" = 1 ] && ! "$RUNTIME" compose version >/dev/null 2>&1; then docker — install the Compose v2 plugin, or upgrade Docker Desktop. podman — it shells out to a compose provider rather than implementing compose itself. Podman Desktop can install one for you, or: brew install podman-compose" + # Before reciting prerequisites, check whether another supported runtime is simply ready. Hit in + # practice with RUNTIME=podman pinned in .env, podman's socket broken, and docker up with compose v2: + # the advice above is all correct and all irrelevant when a working runtime is already installed. + _calt="$(runtime_alternatives_compose)" + for _c in $_calt; do + MISSING="${MISSING} + $_c is installed, responding, and has compose — to use it instead: RUNTIME=$_c ./run.sh + To switch for good, set RUNTIME=$_c in .env (a pinned RUNTIME there overrides auto-detection, + which is the usual reason this message appears while a perfectly good runtime sits unused)." + done + unset _calt _c fi if [ -n "$MISSING" ]; then echo "This kit needs a couple of things that aren't here yet:$MISSING" >&2 diff --git a/scripts/_runtime.sh b/scripts/_runtime.sh index 8597c90..2b33521 100644 --- a/scripts/_runtime.sh +++ b/scripts/_runtime.sh @@ -116,6 +116,29 @@ runtime_alternatives() { printf '%s' "${out# }" } +# _runtime_has_compose -> 0 when that CLI has a v2-style `compose` subcommand. +# +# Its own function, like _runtime_answers, so the policy below is testable without either runtime. +_runtime_has_compose() { + "$1" compose version >/dev/null 2>&1 +} + +# runtime_alternatives_compose -> supported runtimes OTHER than the resolved one that answer AND have +# compose. +# +# Narrower than runtime_alternatives on purpose: a runtime can be reachable and still have no compose +# subcommand, and naming that one as the way out of a compose failure sends the person in a circle. +runtime_alternatives_compose() { + local r out="" + for r in "${_RUNTIME_SUPPORTED[@]}"; do + [ "$r" = "${RUNTIME:-}" ] && continue + _runtime_answers "$r" || continue + _runtime_has_compose "$r" || continue + out="$out $r" + done + printf '%s' "${out# }" +} + runtime_requested() { _runtime_clean "${RUNTIME:-$(_runtime_envv RUNTIME)}"; } # runtime_resolve -> sets and exports RUNTIME; returns 1 with a message on stderr if it cannot. diff --git a/tests/test-runtime.sh b/tests/test-runtime.sh index f87b482..8633470 100755 --- a/tests/test-runtime.sh +++ b/tests/test-runtime.sh @@ -329,6 +329,32 @@ t "alternatives list only runtimes that actually answer" t "alternatives are empty when the only other runtime is installed but dead" [ "$(alts podman 'podman')" = "" ] && ok || bad "got '$(alts podman 'podman')'" +# A compose-capable alternative is a narrower question than a reachable one: a runtime can answer and +# still have no `compose` subcommand, in which case naming it as the way out would send the user in a +# circle. Both probes are separate functions so this is testable with neither runtime installed. +altsc() { # altsc -> runtime_alternatives_compose output + ( export DUPLO_ENV_FILE=/dev/null RUNTIME="$1"; ANS=" $2 "; CMP=" $3 " + . ./scripts/_runtime.sh + eval '_runtime_answers() { case "$ANS" in *" $1 "*) return 0 ;; *) return 1 ;; esac; }' + eval '_runtime_has_compose() { case "$CMP" in *" $1 "*) return 0 ;; *) return 1 ;; esac; }' + runtime_alternatives_compose ) +} + +t "compose alternatives name a runtime that both answers and has compose" +[ "$(altsc podman 'docker podman' 'docker')" = docker ] && ok || bad "got '$(altsc podman 'docker podman' 'docker')'" + +t "compose alternatives exclude a runtime that answers but has no compose" +[ "$(altsc podman 'docker' '')" = "" ] && ok || bad "got '$(altsc podman 'docker' '')' — naming it sends the user in a circle" + +t "compose alternatives never include the runtime we already resolved to" +[ "$(altsc docker 'docker' 'docker')" = "" ] && ok || bad "got '$(altsc docker 'docker' 'docker')'" + +t "run.sh's compose preflight names a compose-capable alternative" +# Hit in practice: .env pinned RUNTIME=podman, podman's socket was broken, docker was up with compose +# v2 — and the failure listed install advice for both runtimes without saying docker would just work. +if grep -B4 -A14 "compose version' failed" run.sh | grep -q 'runtime_alternatives_compose'; then ok +else bad "compose preflight still recites prerequisites instead of naming the working runtime"; fi + # The message itself is pure: given what was resolved, what else answers, and what the toolchain lacks, # it only formats. No probing, so every case below is exercised on any machine. msg() { # msg -> the whole message From 8ff8703342432668466a164af675c7d872346e52 Mon Sep 17 00:00:00 2001 From: Darren Paxton Date: Wed, 30 Sep 2026 15:19:59 -0400 Subject: [PATCH 12/12] fix(runtime): point runtime guidance at .env, never a one-shot on the command line MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every message added here led with `RUNTIME= ./some-script`, which is advice that solves one command and breaks the next. run.sh, build-extension.sh, build-all.sh, logs.sh and stop.sh each source _runtime.sh and resolve independently, so a per-process override does not survive to the following command — and a stack started under one runtime with a build run under another is exactly the cross-runtime condition documented two entries earlier in troubleshooting.md. The guidance was manufacturing the next bug. All three sites now name `.env`, which is the only place every script looks, and say why the command line will not do rather than leaving it as an unexplained preference. They also distinguish the two cases with runtime_requested, because the instruction differs: with RUNTIME already pinned the fix is to change that pin (and the message names its current value, since the pin outranking auto-detection is usually why the person is reading this at all); with nothing pinned the fix is to add it. The one remaining mention of a command-line RUNTIME is in troubleshooting.md, where it explains how a reader ended up with a build and a stack on different runtimes. That is a description of a cause, not a recommendation, and it stays. Tests: 62 (was 60). One asserts no script recommends a per-process override; the other asserts the messages say why .env, so the reasoning cannot be dropped in a later edit while the wording survives. Co-Authored-By: Claude Opus 5 (1M context) --- run.sh | 24 ++++++++++++++++++------ scripts/_builder.sh | 13 ++++++++++--- tests/test-runtime.sh | 12 ++++++++++++ 3 files changed, 40 insertions(+), 9 deletions(-) diff --git a/run.sh b/run.sh index aed10ca..2557ca0 100755 --- a/run.sh +++ b/run.sh @@ -149,13 +149,24 @@ if [ "$RUNTIME_OK" = 1 ] && ! "$RUNTIME" compose version >/dev/null 2>&1; then # practice with RUNTIME=podman pinned in .env, podman's socket broken, and docker up with compose v2: # the advice above is all correct and all irrelevant when a working runtime is already installed. _calt="$(runtime_alternatives_compose)" + _pinned="$(runtime_requested)" for _c in $_calt; do + if [ -n "$_pinned" ]; then + MISSING="${MISSING} + $_c is installed, responding, and has compose. RUNTIME is currently pinned to '$_pinned' — that + pin outranks auto-detection, which would have chosen $_c. Change it in .env: + RUNTIME=$_c" + else + MISSING="${MISSING} + $_c is installed, responding, and has compose. Set it in .env: + echo 'RUNTIME=$_c' >> .env" + fi MISSING="${MISSING} - $_c is installed, responding, and has compose — to use it instead: RUNTIME=$_c ./run.sh - To switch for good, set RUNTIME=$_c in .env (a pinned RUNTIME there overrides auto-detection, - which is the usual reason this message appears while a perfectly good runtime sits unused)." + Set it in .env rather than per-command: run.sh, the build scripts, logs.sh and stop.sh each + resolve the runtime independently, so a one-off on the command line leaves the next command on + the other runtime." done - unset _calt _c + unset _calt _pinned _c fi if [ -n "$MISSING" ]; then echo "This kit needs a couple of things that aren't here yet:$MISSING" >&2 @@ -170,8 +181,9 @@ if ! "$RUNTIME" info >/dev/null 2>&1; then _alt="$(runtime_alternatives)" if [ -n "$_alt" ]; then for _a in $_alt; do - echo " $_a is installed and responding. To use it instead: RUNTIME=$_a ./run.sh" >&2 - echo " (or make it this checkout's default: echo 'RUNTIME=$_a' >> .env)" >&2 + echo " $_a is installed and responding. To use it, set it in .env — every script reads it" >&2 + echo " there, and a one-off on the command line would leave the next command elsewhere:" >&2 + echo " echo 'RUNTIME=$_a' >> .env" >&2 done fi unset _alt _a diff --git a/scripts/_builder.sh b/scripts/_builder.sh index 55a4dfc..aae8ea1 100644 --- a/scripts/_builder.sh +++ b/scripts/_builder.sh @@ -170,12 +170,19 @@ builder_runtime_message() { echo "" >&2 # shellcheck disable=SC2086 # deliberate word-split: alts is a space-separated list for a in $alts; do - echo " $a is installed and responding. Use it for this build:" >&2 + if [ "$requested" = 1 ]; then + echo " $a is installed and responding. RUNTIME is pinned to '$rt' — change it in .env:" >&2 + else + echo " $a is installed and responding. Set it in .env:" >&2 + fi echo "" >&2 - echo " RUNTIME=$a ./scripts/build-extension.sh " >&2 + echo " RUNTIME=$a" >&2 echo "" >&2 - echo " ...or make it this checkout's default: echo 'RUNTIME=$a' >> .env" >&2 + echo " In .env and not on the command line: run.sh, the build scripts, logs.sh and stop.sh" >&2 + echo " each resolve the runtime independently, so a one-off would leave the next command on" >&2 + echo " the other runtime — which is how a build ends up unable to see the running studio." >&2 done + echo "" >&2 echo " ...or start $rt and re-run." >&2 else echo "" >&2 diff --git a/tests/test-runtime.sh b/tests/test-runtime.sh index 8633470..d32a89e 100755 --- a/tests/test-runtime.sh +++ b/tests/test-runtime.sh @@ -349,6 +349,18 @@ t "compose alternatives exclude a runtime that answers but has no compose" t "compose alternatives never include the runtime we already resolved to" [ "$(altsc docker 'docker' 'docker')" = "" ] && ok || bad "got '$(altsc docker 'docker' 'docker')'" +t "runtime guidance points at .env, never a one-shot on the command line" +# A CLI-scoped RUNTIME lasts exactly one process. run.sh, build-extension.sh, build-all.sh, logs.sh and +# stop.sh each resolve independently, so a one-off gets you past one command and leaves the next on the +# other runtime — which is precisely how the cross-runtime build warning happens. .env is the only place +# every script reads, so it is the only honest advice. +HITS="$(grep -nE "RUNTIME=[^ \"']+ +\./" run.sh scripts/_builder.sh scripts/_runtime.sh 2>/dev/null)" +if [ -z "$HITS" ]; then ok; else bad "recommends a per-process override: $HITS"; fi + +t "runtime guidance says why .env and not per-command" +if grep -qiE 'each script resolves|every script reads|resolve.*independently' run.sh scripts/_builder.sh; then ok +else bad "tells the user where to set it but not why the command line will not do"; fi + t "run.sh's compose preflight names a compose-capable alternative" # Hit in practice: .env pinned RUNTIME=podman, podman's socket was broken, docker was up with compose # v2 — and the failure listed install advice for both runtimes without saying docker would just work.