diff --git a/.env.example b/.env.example index c8de660..6eb4b14 100644 --- a/.env.example +++ b/.env.example @@ -22,12 +22,35 @@ XTERM_TAG=dev-0.1.2-6cdf181 QDRANT_PORT=6333 QDRANT_API_KEY=devkit-local-qdrant +# ───────────────────────────────────────────────────────────────────────────── +# Container runtime — which CLI runs the stack and the builds. +# +# Leave it commented out to auto-detect, which tries docker then podman, taking the first one +# INSTALLED (presence only — it is not checked for being reachable), so a machine that has always had +# Docker keeps behaving exactly as before. Installing Docker alongside podman therefore moves you onto +# Docker silently. Set it to pin one: +# +# docker | podman +# +# The requirement is a Docker-compatible CLI that also has a `compose` subcommand — podman delegates +# that to podman-compose or docker-compose, so install one of those alongside it. +# +# NOT a valid value: runc (or crun, youki, runsc…). Those are low-level OCI runtimes that execute an +# already-unpacked bundle given by path — they have no images, registries, networks or compose, so +# there is no `runc run `. They are what the CLIs above call underneath; to choose one, tell +# your CLI (e.g. `podman --runtime crun`), not this variable. +# +# Rootless podman is handled automatically: builds run as uid 0 inside the user namespace, which is +# what makes the extension bundle come out owned by you rather than by an unreachable subuid. +# ───────────────────────────────────────────────────────────────────────────── +#RUNTIME=docker + # Extension build toolchain (.NET SDK 8 + Node 22). Only pulled the first time you build an # extension — the platform itself doesn't use it. An already-pulled tag is reused without checking the # registry: BUILDER_PULL=1 forces a re-pull (this tag moves), BUILDER_IMAGE= replaces the image # outright, BUILDER_PLATFORM= forces an architecture. Each build's package caches live in -# per-uid named volumes that nothing prunes — `docker volume ls | grep duplo_devkit` to find and -# `docker volume rm` to reclaim them (each holds a full NuGet + npm closure). +# per-uid named volumes that nothing prunes — ` volume ls | grep duplo_devkit` to find and +# ` volume rm` to reclaim them (each holds a full NuGet + npm closure). BUILDER_TAG=latest # Build on THIS machine instead of in the toolchain container. Set to 1 if you already have .NET SDK 8 @@ -37,6 +60,15 @@ BUILDER_TAG=latest # same thing for a single run, and wins over this. Leave it commented out to build in the container. #DUPLO_BUILD_NATIVE=1 +# Minimum podman VM size run.sh will start on (macOS/Windows only — a native-Linux podman has no VM, +# and docker is not checked). The VM's memory is fixed at create time and shared by this +# stack AND every extension build; too little and the Angular frontend build is OOM-killed, reporting +# itself only as 'exit status 137' at the foot of a Go stack trace. Undersized memory is a hard failure; +# a low CPU count is just a warning. Lower the floor only if you know a smaller machine suits you — +# 2 GiB is known to fail, 8 GiB is comfortable. +#PODMAN_MIN_MEMORY_MIB=6144 +#PODMAN_MIN_CPUS=4 + # On Apple Silicon the studio image is amd64-only (emulated). Leave as-is unless you have an arm64 image. STUDIO_PLATFORM=linux/amd64 diff --git a/README.md b/README.md index 7591daa..7eb3a64 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ [![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE) [![Release](https://img.shields.io/github/v/release/duplocloud/devkit?label=release)](https://github.com/duplocloud/devkit/releases) -Run the DuploCloud AI HelpDesk platform on your laptop via Docker — then build your own +Run the DuploCloud AI HelpDesk platform on your laptop in containers — then build your own **Agent** against it. Each Agent you write ships a backend, a UI, and its own provisioning, and hot-loads into the running platform with no restart. @@ -20,7 +20,8 @@ hot-loads into the running platform with no restart. ## Quick start -You need **Docker with Compose v2**, **Python 3**, and access to an LLM. `./run.sh` prompts for +You need **a container runtime with Compose v2** (docker or podman — set `RUNTIME` +in `.env` to pin one, otherwise it is auto-detected), **Python 3**, and access to an LLM. `./run.sh` prompts for Anthropic, AWS Bedrock, or an Anthropic-compatible LLM gateway (OpenRouter, Bifrost, LiteLLM, …). ```bash diff --git a/docker-compose.yml b/docker-compose.yml index b3e32fd..d3c3c78 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -157,34 +157,45 @@ services: environment: - CORS_ORIGIN=* - # Build toolchain for extensions (.NET SDK 8 + Node 22). Behind a profile so `docker compose pull` + # Build toolchain for extensions (.NET SDK 8 + Node 22). Behind a profile so the `compose pull` # and `up` in run.sh skip it entirely — a user who only runs the platform never downloads it. The - # build wrapper starts it on demand with `docker compose run --rm --no-deps builder`. + # build wrapper starts it on demand with `$RUNTIME compose run --rm --no-deps builder`. # - # Being a compose service (rather than a bare `docker run`) is what lets the build reach the studio over + # Being a compose service (rather than a bare `run`) is what lets the build reach the studio over # the compose network — the SDK feed comes from AIStudio at http://duplo-ai-studio:60021 — but only when - # the studio is in THIS compose project. `docker compose` is project-scoped (the project defaults to the + # the studio is in THIS compose project. Compose is project-scoped (the project defaults to the # directory name), so a platform started from a second checkout or a git worktree — which is how this # repo is developed — is not on this network and that name does not resolve. scripts/_builder.sh detects # which case it is and falls back to the published host port via host.docker.internal (see extra_hosts # below). Both routes are live; the compose network is simply the preferred one. builder: profiles: [tools] - image: ${BUILDER_IMAGE:-quay.io/duplocloud/duplo-extension-builder:${BUILDER_TAG:-latest}} + # Deliberately ONE level of interpolation, not a nested ${BUILDER_IMAGE:-…:${BUILDER_TAG:-latest}}: + # podman-compose 1.5 stops at the first '}' and appends the remainder literally, yielding + # "…busybox:1.36}" and an "invalid reference format" error even when BUILDER_IMAGE is set. Nothing + # is lost — BUILDER_TAG is resolved into BUILDER_IMAGE by scripts/_builder.sh:builder_resolve_image, + # which always exports it before compose is invoked, so the default below is only ever reached by a + # hand-run `compose run` with no BUILDER_IMAGE set (where it now means :latest regardless of + # BUILDER_TAG — use the build scripts, or set BUILDER_IMAGE outright, to pin another tag). + image: ${BUILDER_IMAGE:-quay.io/duplocloud/duplo-extension-builder:latest} # Unset by default, so the build runs on the host's native architecture. Set BUILDER_PLATFORM only # to force one (e.g. linux/amd64 if a publish ever ends up amd64-only) — an unset value is dropped # by compose, whereas copying duplo-ai-studio's linux/amd64 default would silently emulate a .NET # build on Apple Silicon. platform: ${BUILDER_PLATFORM:-} - # The caller's real uid/gid, exported by scripts/_builder.sh — so dist/ ends up owned by the - # invoking user, not root. The default is only a fallback for a hand-run `docker compose run`. + # The ids that make dist/ come out owned by the invoking user, exported by scripts/_builder.sh. + # NOT always `id -u`: under rootless podman the caller is already mapped to uid 0 inside the user + # namespace, so 0:0 is what yields caller-owned files and a real uid maps to an unwritable subuid. + # scripts/_runtime.sh:runtime_builder_ids picks per runtime. The default here is only a fallback for + # a hand-run `compose run`, and is correct for docker; under rootless podman pass DUPLO_UID=0. user: "${DUPLO_UID:-1000}:${DUPLO_GID:-1000}" working_dir: /work # Lets the build reach a studio that is NOT in this compose project via the published host port. # compose scopes services per project (the directory name), so a stack started in another checkout # is not on this project's network and duplo-ai-studio does not resolve; _builder.sh detects that and # points DUPLO_BASE at host.docker.internal instead. A no-op on Docker Desktop, where the name - # already resolves; on Linux it is what makes it resolve at all. + # already resolves; on Linux (and under podman, which maps host-gateway the same way) it is what + # makes it resolve at all. extra_hosts: - "host.docker.internal:host-gateway" volumes: diff --git a/docs/configuration.md b/docs/configuration.md index 30c5f2a..978d6b4 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -8,6 +8,7 @@ Everything the dev kit reads comes from `.env` in the repo root. Copy it from `. > [SUPPORT.md](../SUPPORT.md) before sharing any output. - [How `.env` is maintained](#how-env-is-maintained) +- [Container runtime](#container-runtime) - [Licensing](#licensing) - [Image tags](#image-tags) - [Host ports](#host-ports) @@ -27,6 +28,45 @@ Framework-shipped defaults — the image tags, the studio platform, and the lice your `.env` **only if you have not diverged from the previously-applied default**. A tag you pinned is always kept. The mechanism is described in [upgrading.md](upgrading.md#how-defaults-are-adopted). +## Container runtime + +| Variable | Default | Effect | +| --- | --- | --- | +| `RUNTIME` | *unset* — auto-detect | Which container CLI every script drives. Accepts `docker` or `podman`. | + +Docker is the default and needs no configuration. Left unset, the scripts auto-detect by walking the +supported CLIs in order and taking the first one found. + +Two things about that detection are worth knowing, because both have cost people time: + +- **It tests presence only — there is no daemon check.** A CLI that is installed but not running still + wins the detection. This is deliberate (`scripts/_runtime.sh`): conflating "installed" with "working" + would make an unstarted Docker Desktop silently fall through to another runtime, and a build would + then run somewhere you did not intend. +- **Order is precedence.** Installing Docker alongside an existing podman silently moves you onto Docker. + A correctly configured podman machine can be up and idle while `./run.sh` uses Docker instead. + +Check which one you will actually get, and pin it if you care: + +```bash +bash -c 'source scripts/_runtime.sh; echo "using: $(runtime_detect)"' +``` + +```bash +RUNTIME=podman # docker | podman +``` + +An explicitly requested runtime that turns out to be unusable is a **hard error**, not a fallback — +silently building with something other than what you asked for is never right. Auto-detection finding +nothing may still fall back to a native toolchain build. + +`RUNTIME` does not select a *low-level* OCI runtime. `runc`, `crun`, `youki` and `runsc` are not valid +values — they run an already-unpacked bundle by path and have no images, registries or compose. Choose +one through your CLI instead (`podman --runtime crun`). + +See [prerequisites](getting-started/prerequisites.md#01-a-container-runtime-with-compose-v2) for +install-time requirements, including the podman-specific setup on Apple Silicon. + ## Licensing The dev kit is licensed, and the studio will not serve without a license. `run.sh` handles this on first diff --git a/docs/getting-started/prerequisites.md b/docs/getting-started/prerequisites.md index e086dc2..d4d1bc9 100644 --- a/docs/getting-started/prerequisites.md +++ b/docs/getting-started/prerequisites.md @@ -1,23 +1,24 @@ # 0. Prerequisites -**What you'll do:** Confirm you have what this kit needs — Docker with Compose v2, Python 3, LLM access, -a verified email address, and five free ports. +**What you'll do:** Confirm you have what this kit needs — a container runtime with Compose v2, Python 3, +LLM access, a verified email address, and five free ports. **What you need first:** Nothing. This is the first page. The dev kit runs the whole platform from published images. There is no *language* toolchain to install — no .NET, no Node: every container it starts, and every extension you later build, is built and run inside -Docker. The two things that must exist on the host are Docker itself and `python3`, which the setup -scripts use as their JSON and `.env` editor. +a container. The two things that must exist on the host are a container runtime and `python3`, which the +setup scripts use as their JSON and `.env` editor. --- -## 0.1 Docker, with Compose v2 +## 0.1 A container runtime, with Compose v2 -Docker Desktop, Colima, Rancher Desktop, or a plain Docker Engine all work. What matters is that the -daemon is running and that `docker compose` (two words, the v2 plugin) exists. +Docker is the default and needs no configuration. **podman** is also supported — Docker Desktop, +Colima, Rancher Desktop, plain Docker Engine and rootless podman all work. +What matters is that the runtime is reachable and that its `compose` subcommand (two words) exists. -1. Check both at once. +1. Check both at once — substitute your runtime for `docker`. ```bash docker --version && docker compose version @@ -33,6 +34,73 @@ daemon is running and that `docker compose` (two words, the v2 plugin) exists. Any Compose `v2.x` or newer is fine. If the second line does not print, you have Compose v1 only and need the v2 plugin. +2. If you have more than one runtime installed, or want to pin one, set `RUNTIME` in `.env`: + + ```bash + RUNTIME=podman # docker | podman + ``` + + Left unset, the scripts auto-detect, trying `docker` then `podman`, and taking the first one + **installed** — presence only, with no check that it is running. Installing Docker alongside podman + therefore moves you onto Docker silently. Confirm which you will get: + + ```bash + bash -c 'source scripts/_runtime.sh; echo "using: $(runtime_detect)"' + ``` + +> **podman:** it needs a compose provider, because podman shells out to one rather than implementing +> compose itself. Podman Desktop installs one during setup (a `docker-compose` binary, which podman +> delegates to), or `brew install podman-compose`. Either satisfies the `podman compose version` check +> `run.sh` runs — so if that already answers, there is nothing to install. Rootless +> podman needs nothing else; the build scripts handle the user-namespace uid mapping for you, so +> extension bundles come out owned by you rather than by root. +> +> **podman on Apple Silicon** needs three more things before the studio will start, because the studio +> image is amd64 on some tags and runs emulated: +> +> 1. **Create the machine with `--provider applehv`.** Rosetta is an applehv feature. podman 6 defaults +> to **libkrun** on Apple Silicon, and libkrun has no Rosetta support at all — upstream's +> `LibKrunStubber.GetRosetta` returns false unconditionally, so a `rosetta = true` under libkrun is +> read and silently discarded, and `podman machine inspect` keeps reporting `Rosetta: false` however +> many times you edit the config. **Unlike the rosetta key, the provider is fixed at +> `podman machine init`** — it is the one setting an existing machine cannot be talked out of, so +> switching means destroying and recreating the VM. +> 2. **Enable Rosetta.** podman does **not** enable it by default, and without it amd64 binaries fall +> through to QEMU, which cannot run the studio's .NET runtime. Create +> `~/.config/containers/containers.conf` with `[machine] provider = "applehv"` and +> `[machine] rosetta = true` before starting the machine (on applehv the rosetta key is re-read on +> every `podman machine start`, so *that* key only needs a stop/start, not re-creating). **If that +> file already exists, edit its `[machine]` section — never append a second `[machine]` table, which +> is a TOML duplicate-key error that stops podman running at all.** +> 3. **Size the machine.** `podman machine init` defaults to 2048 MiB; `run.sh` hard-fails below +> 6144 MiB, because an undersized VM does not stop the stack coming up — it poisons later extension +> builds with an OOM kill that reports itself as `exit status 137`. +> +> Which makes the whole first-time sequence: +> +> ```bash +> mkdir -p ~/.config/containers +> printf '[machine]\nprovider = "applehv"\nrosetta = true\n' > ~/.config/containers/containers.conf +> podman machine init --provider applehv --cpus 4 -m 8192 --disk-size 100 +> podman machine start +> ``` +> +> Verify all three before your first run: +> +> ```bash +> podman machine list --format '{{.Name}} {{.VMType}}' # want applehv, NOT libkrun +> podman machine ssh 'ls /proc/sys/fs/binfmt_misc/' # want a `rosetta` entry, no `qemu-x86_64` +> podman machine list # want MEMORY >= 6144 MiB +> ``` +> +> If the studio hangs on startup anyway, see +> [troubleshooting](../troubleshooting.md#runsh-hangs-on-waiting-for-studio-apple-silicon). +> +> **Not a runtime:** `runc` (and `crun`, `youki`, `runsc`). Those are low-level OCI runtimes that run an +> already-unpacked bundle by path — they have no images, registries or compose, so they cannot drive +> this kit. They are what the CLIs above use underneath; select one via your CLI, e.g. +> `podman --runtime crun`. + ## 0.2 Python 3 `run.sh` and everything under `scripts/` shell out to `python3` to rewrite `.env` safely (tokens and keys @@ -46,7 +114,8 @@ python3 --version macOS ships one with the Xcode command line tools (`xcode-select --install`) or `brew install python3`; Debian/Ubuntu, `sudo apt-get install -y python3`; RHEL/Amazon Linux, `sudo dnf install -y python3`. -`./run.sh` checks for this and for Docker before it does anything else, and names whatever is missing. +`./run.sh` checks for this and for your container runtime before it does anything else, and names +whatever is missing. ## 0.3 LLM access diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 92b89cd..9460913 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -32,8 +32,10 @@ This kit needs a couple of things that aren't here yet: ``` **Why** — `run.sh` checks its two host prerequisites before it touches anything: `python3` (it is the -`.env` editor and the JSON parser for every API call the setup makes) and `docker`, including the -`docker compose` v2 *subcommand* — the standalone `docker-compose` v1 binary does not satisfy it. Every +`.env` editor and the JSON parser for every API call the setup makes) and a container runtime +(`docker` or `podman` — auto-detected, or pinned with `RUNTIME` in `.env`), +including its `compose` v2 *subcommand* — the standalone `docker-compose` v1 binary does not satisfy +it, and podman needs a compose provider installed alongside it (`podman-compose`). Every missing item is listed in one pass, so the list is the whole list. **Fix** — install what it names, per [prerequisites](getting-started/prerequisites.md), and re-run. @@ -45,8 +47,8 @@ missing item is listed in one pass, so the list is the whole list. python3 --version && docker compose version ``` -A *stopped* Docker daemon is only a warning at this stage; it becomes a real error later, at -`docker compose pull`. +An unreachable runtime is only a warning at this stage; it becomes a real error later, at +`compose pull`. ### A port is already in use @@ -62,7 +64,7 @@ but a port can still be taken by something else on your machine. The defaults ar `UI_PORT=4210`, `AGENT_PORT=8010`, `MONGO_PORT=27018`, `XTERM_PORT=6061`, `QDRANT_PORT=6333`. `QDRANT_PORT` is the exception to the offset: 6333 is Qdrant's own default, so a Qdrant you already run -locally will hold it. `docker compose up -d` is not guarded against this — one clashing port aborts the +locally will hold it. `compose up -d` is not guarded against this — one clashing port aborts the whole run — so move it in `.env` before starting. **Fix** — find what holds the port, then either stop it or move the dev kit: @@ -82,10 +84,232 @@ docker compose ps # every service Up curl -fsS http://localhost:4210 >/dev/null && echo "UI is answering" ``` +### podman-specific + +All of these are handled by the scripts; they are listed because you will meet them the moment you run +a `compose` command *by hand* under podman, and the errors do not say what is actually wrong. + +| Symptom | Cause | What to do | +| --- | --- | --- | +| `missing services [builder]` | podman-compose does not implicitly enable a service's `profiles` the way `docker compose run` does, and `builder` sits behind the `tools` profile. `COMPOSE_PROFILES=tools` does **not** help — podman-compose ignores that variable. | Name it: `podman compose --profile tools run --rm --no-deps builder …` | +| `invalid reference format` on an image ending in `}` | podman-compose 1.5 stops at the first `}` of a nested `${A:-${B}}` and appends the rest literally. | Already removed from `docker-compose.yml`; don't reintroduce nesting — `tests/test-runtime.sh` checks for it. | +| `compose ps` exits 2 with a usage dump | podman-compose's `ps` has no `--services` or `--status`; its whole grammar is `-q` and `--format`. | The scripts use `runtime_compose_running_services` (in `scripts/_runtime.sh`), which falls back to filtering on the `com.docker.compose.*` labels that both implementations set. | +| An extension bundle in `dist/` is owned by uid 101000 and you cannot delete it | Rootless podman maps *your* uid to container uid 0, so passing a real `--user 1000` lands on an unwritable subuid. | The build scripts pass `0:0` under rootless podman automatically. For an already-broken directory: `podman unshare rm -rf dist`. | +| `rootless netns: kill network process: permission denied` on container teardown | A podman rootless cleanup quirk. Cosmetic — it is printed after the container has already exited, and the exit status is still propagated correctly. | Ignore it. | + +`host.docker.internal` works under podman: the `extra_hosts: host-gateway` entry in `docker-compose.yml` +maps it, and podman additionally provides `host.containers.internal` for the same address. + +### `./run.sh` hangs on "Waiting for studio" (Apple Silicon) + +> `run.sh` now catches this before it happens — `runtime_rosetta_check` in `scripts/_runtime.sh` stops +> the run with the fix below rather than letting it hang. You should only reach this entry if the check +> could not reach the machine to judge it (it stays silent rather than guessing), or if you are on a +> version of the kit that predates it. The diagnosis and fix are the same either way. + +**Symptom** — minutes of dots, then a failure that blames the wrong thing: + +``` +==> Waiting for studio at http://localhost:60031 ........................... +==> Minting permanent admin API token… +Login failed for you@example.com — check the admin email/password (./run.sh --reset to re-enter). +``` + +The studio container looks healthy — `podman ps` reports `Up`. Its log does not: + +``` +at Program.
$(String[] args) in /src/Duplo.ai.studio/Program.cs:line 255 +qemu: uncaught target signal 6 (Aborted) - core dumped +``` + +**Why** — feature-branch studio images are built amd64-only (`platform: ${STUDIO_PLATFORM:-linux/amd64}` +in `docker-compose.yml`), so on Apple Silicon the studio runs emulated. There are two emulators and only +one of them works: + +- **Rosetta** — used when the podman VM has the Rosetta share attached. Runs .NET correctly. +- **QEMU user-mode** — the fallback. Cannot run .NET's JIT/reflection; the process takes `SIGABRT` while + `MapControllers()` walks controller attributes at startup. + +**First check the provider — under libkrun none of the rest of this applies.** podman 6 defaults to the +`libkrun` provider on Apple Silicon, and libkrun has **no Rosetta support at all**: upstream's +`LibKrunStubber.GetRosetta` returns false unconditionally, so `rosetta = true` is read and discarded +without a warning anywhere. The symptom is identical to a missing key — no `rosetta` binfmt handler, QEMU, +a studio that aborts — but every fix aimed at the config or at restarting the machine is wasted, because +the config was never the problem: + +```bash +podman machine list --format '{{.Name}} {{.VMType}}' # want applehv; libkrun can never have Rosetta +``` + +Unlike the rosetta key, **the provider is fixed at `podman machine init`**. Switching to applehv means +recreating the VM, which destroys its images and volumes: + +```bash +podman machine stop && podman machine rm -f +podman machine init --provider applehv --cpus 4 -m 8192 --disk-size 100 +podman machine start +``` + +Podman Desktop muddies this further: it has a known bug ([#16341](https://github.com/podman-desktop/podman-desktop/issues/16341)) +where it reports Rosetta as enabled on libkrun machines that cannot have it. + +**On applehv, Rosetta is opt-in, and podman does not enable it for you.** Unless +`~/.config/containers/containers.conf` sets `[machine] rosetta = true`, the VM is started with no Rosetta +share and every amd64 binary falls through to QEMU. The key is read on every `podman machine start`, not +only at `init`. Measured on an **applehv** machine, podman 6.1.2, reading `AppleHypervisor.Vfkit.Rosetta` +from the machine's JSON (these numbers do not describe libkrun, which is false in every row): + +| `containers.conf` | resulting `Vfkit.Rosetta` | +| --- | --- | +| `[machine] rosetta = true` | `true` | +| *file absent* | `false` | +| `[machine] rosetta = false` | `false` | + +An absent file behaves exactly like an explicit `false`. Do not assume the default is on. + +Nothing tells you it is off, because it then fails **silently** twice over: + +1. Inside the VM, `rosetta-activation.service` runs `mount -t virtiofs rosetta /var/mnt`, fails with + `wrong fs type, bad option, bad superblock on rosetta`, and **exits 0 anyway**. `systemctl status` + reports the unit as `status=0/SUCCESS`. +2. The studio container stays `Up` after the .NET process aborts — the crash does not take PID 1 down + promptly — so `podman ps` shows nothing wrong. + +A second possible cause, **not observed here and not verified** — recorded because the strings are present +in the `vfkit` binary and it would produce identical symptoms even with the config set correctly: vfkit +degrades on its own with `Rosetta installation failed. Continuing without Rosetta.` and still exits 0, +which would happen if Rosetta were not registered on the host when the machine starts and vfkit probes +for it. The diagnosis below catches it either way; step 1 of the fix rules it out. + +With the studio dead, `/healthz` never answers and the 90 × 3s poll in `run.sh` spins for ~4.5 minutes +before falling through to a login failure that has nothing to do with your password. + +> **Do not trust the top-level `Rosetta` field** in `~/.config/containers/podman/machine/applehv/podman-machine-default.json`. +> It is not the field that decides, and it is routinely the *opposite* of the effective value. The one +> that matters is `AppleHypervisor.Vfkit.Rosetta`, which is what `podman machine inspect` reports. + +**Diagnose** — the decisive check is the binfmt handler list inside the VM: + +```bash +podman machine ssh 'ls /proc/sys/fs/binfmt_misc/' +``` + +| You see | Meaning | +| --- | --- | +| `rosetta` present | Rosetta is live. This is not your problem — look elsewhere. | +| `qemu-x86_64` present, no `rosetta` | Broken. amd64 runs under QEMU and .NET will abort. | + +Corroborate: + +```bash +podman machine list --format '{{.VMType}}' # libkrun means Rosetta is impossible, not absent +podman machine inspect --format '{{.Rosetta}}' # false when broken +podman machine ssh 'mount | grep -i rosetta' # no output when broken +podman machine ssh 'systemctl status rosetta-activation.service --no-pager -l' + # "wrong fs type" despite status=0/SUCCESS +``` + +**Fix** — add the config, then restart the machine. **The machine does not need to be recreated.** podman +reads `[machine] rosetta` at every `podman machine start`, not only at `init`, so a stop/start picks it up +and nothing is destroyed. + +1. Confirm Rosetta on the host first — the VM share cannot be created if the host has no Rosetta: + +```bash +arch -x86_64 /usr/bin/true && echo "Rosetta OK" || softwareupdate --install-rosetta --agree-to-license +``` + +If you just installed it, re-run the `arch -x86_64` check and see it succeed before going on. + +2. Opt in to Rosetta. **This is mandatory, not a nicety** — see the table above. + + **Only if `~/.config/containers/containers.conf` does not exist yet:** + +```bash +mkdir -p ~/.config/containers +printf '[machine]\nrosetta = true\n' > ~/.config/containers/containers.conf +``` + + > ⚠️ **If the file already exists, edit it — do not append.** Add `rosetta = true` under its + > existing `[machine]` section, or add a `[machine]` section if it has none. Appending a second + > `[machine]` table is a TOML duplicate-key error and podman then refuses to run **at all**: + > + > ``` + > Failed to obtain podman configuration: parsing containers.conf: + > toml: line 11: Key 'machine' has already been defined. + > ``` + > + > That is a worse state than the missing Rosetta you started with. `run.sh`'s preflight reads your + > actual config and prints the right instruction for whichever case you are in. + + Keep the file — it is read on every start, so deleting it silently loses Rosetta at the next one. + +3. Restart the machine. No `rm`, no `init`, no re-pull — images and volumes are untouched: + +```bash +podman machine stop +podman machine start +``` + +> Verified by toggling one machine twice: with the file absent a stop/start yields `Vfkit.Rosetta: false` +> and a `qemu-x86_64` binfmt handler; with the file present the same machine yields `true` and a `rosetta` +> handler. The machine's `Created` timestamp is unchanged throughout, so no re-creation is involved. +> +> Only recreate the machine if you need to change its **sizing** — and then pass your sizing back +> explicitly, because the `init` default is 2048 MiB and `runtime_machine_check` rejects anything under +> 6144: `podman machine init --cpus 4 -m 8192 --disk-size 100`. + +**Confirm** — run this gate *before* `./run.sh`, so a failure costs seconds rather than a confusing hang: + +```bash +podman machine inspect --format '{{.Rosetta}}' # true +podman machine ssh 'ls /proc/sys/fs/binfmt_misc/' | grep rosetta # rosetta +podman machine ssh 'mount | grep -i rosetta' # rosetta on /var/mnt type virtiofs +podman run --rm --platform linux/amd64 docker.io/library/alpine:3 uname -m # x86_64 +``` + +The first three must pass, and `qemu-x86_64` should now be **absent** from the binfmt list — Rosetta +replaces it. The fourth is a liveness check only: a trivial amd64 binary runs under QEMU too, so `x86_64` +there confirms amd64 executes but does **not** prove Rosetta is the thing executing it. The binfmt list is +the discriminating check. +Then start the stack and verify the studio is alive rather than merely `Up`: + +```bash +./run.sh +curl -fsS http://localhost:60031/healthz && echo " studio is answering" +podman logs devkit-duplo-ai-studio-1 2>&1 | grep -c 'uncaught target signal' # must be 0 +``` + +**Avoiding it entirely** — if an arm64 studio image exists for your tag, use it and skip emulation. +Observed so far: `branch-*` tags are published amd64-only, while `dev-*` tags are multi-arch. Never assume +either way — check: + +```bash +podman manifest inspect "quay.io/duplocloud/backend:$(grep '^STUDIO_TAG=' .env | cut -d= -f2)" \ + | grep architecture +``` + +If an `arm64` variant is listed, set `STUDIO_PLATFORM=linux/arm64` in `.env` and re-run `./run.sh`. + +**Or use Docker instead** — Docker Desktop on Apple Silicon provides Rosetta itself, with no +`containers.conf` and no machine configuration. The same amd64 studio image that aborts under podman's +QEMU starts normally under Docker: `uname -m` reports `x86_64` and the log carries zero +`uncaught target signal` entries. This whole entry is podman-specific. + +> **If you have both installed, check which one you are actually using.** `runtime_detect` in +> `scripts/_runtime.sh` walks `docker podman` and takes the first on `PATH` — presence only, with no +> daemon check — so installing Docker silently takes precedence over podman. A podman machine can be up +> and correctly configured while `./run.sh` uses Docker. Pin it deliberately: +> +> ```bash +> bash -c 'source scripts/_runtime.sh; echo "using: $(runtime_detect)"' # what it will pick +> echo 'RUNTIME=podman' >> .env # force podman +> ``` + ### More entries belong here -Image pull failures and `docker login quay.io`, and insufficient memory or disk. Add them as they are hit — -with the verbatim error string. +Image pull failures and ` login quay.io`, and insufficient memory or disk. Add them as they are +hit — with the verbatim error string. --- @@ -345,7 +569,7 @@ up and reachable at the resolved base URL before a build can start. With `DUPLO_ - **Local** — confirm the stack is running and the port matches `.env`: ```bash - docker compose ps + ./logs.sh --no-follow duplo-ai-studio # or: compose ps grep STUDIO_PORT .env ``` @@ -364,6 +588,45 @@ A version string like `{"version":"1.0.6"}` means the build will get past this s **authenticated** — without the header you get a 401, which is not the same problem as the studio being down. For that, `curl -fsS http://localhost:60031/healthz` answers anonymously. +### The build warns it cannot see the studio, then works anyway + +**Symptom** — `./scripts/build-extension.sh` prints a warning, then the build carries on and succeeds: + +``` +WARNING: no running duplo-ai-studio in this compose project (docker compose scopes services + per project) … will try the published host port instead: + http://host.docker.internal:60031 +``` + +**Why** — the build and the stack are on **different container runtimes**. Compose scopes services per +project *per runtime*, so a build running under docker cannot see a studio that was started under podman, +or the reverse. It is easy to end up here because detection is presence-based and docker-first: you may +have started the stack with `RUNTIME=podman ./run.sh` while the build, run without that variable, +auto-detects docker. + +**This is usually benign.** The fallback URL reaches the studio through its *published host port* — the +same port your browser uses — so the SDK fetch still succeeds and the bundle builds correctly. Treat it as +noise unless the build actually fails on the SDK step, which is the previous entry. + +**Confirm the fallback route is alive** — from inside a container on the build's runtime: + +```bash +RT=docker # the runtime the BUILD uses, not the stack's +IMG=$(grep ^BUILDER_IMAGE= .env | cut -d= -f2-) +PORT=$(grep ^STUDIO_PORT= .env | cut -d= -f2-); PORT=${PORT:-60021} +$RT run --rm --add-host host.docker.internal:host-gateway "$IMG" \ + curl -sS -o /dev/null -w '%{http_code}\n' \ + "http://host.docker.internal:$PORT/v1/aiservicedesk/extensions/sdk-version" +``` + +**`401` means the route is fine.** The SDK feed is authenticated, so an unauthenticated request from inside +the container is *supposed* to be rejected — reaching a 401 proves the network path works. A connection +error or `000` is the real failure. Do not read the 401 as the problem. + +**Silence it** — put both on the same runtime, e.g. `echo 'RUNTIME=podman' >> .env`, so the build and the +stack share a compose project. The builder image is cached per runtime, so the first build after switching +pays a one-time pull. + ### The extension's page never loads in the portal The portal loads extension remotes with Native Federation and reads a `remoteEntry.json`. Check, in diff --git a/logs.sh b/logs.sh index 5394c5a..5ac30d7 100755 --- a/logs.sh +++ b/logs.sh @@ -2,4 +2,6 @@ # Tail logs. Optional service name: duplo-ai-studio | claude-code-agent | duplo-ui | mongo | xterm set -euo pipefail cd "$(dirname "$0")" -exec docker compose logs -f "$@" +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman) +runtime_resolve || exit 1 +exec "$RUNTIME" compose logs -f "$@" diff --git a/run.sh b/run.sh index 410fab3..2557ca0 100755 --- a/run.sh +++ b/run.sh @@ -50,6 +50,7 @@ set -euo pipefail cd "$(dirname "$0")" ENV=.env +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman); resolved in preflight . ./scripts/_provider_gateway.sh . ./scripts/_provider_subscription.sh . ./scripts/_studio_api.sh @@ -101,6 +102,14 @@ while [ $# -gt 0 ]; do shift done +# ── .env must exist before anything reads it ────────────────────────────────── +# Ahead of the prerequisite checks deliberately: runtime_resolve below reads RUNTIME from .env and then +# memoises its answer for the whole run. Creating .env after that point meant a first run on a box with +# both CLIs installed auto-detected docker and ignored RUNTIME=podman — set in .env or shipped in +# .env.example — for both podman preflights, `compose pull` and `compose up`. Needs neither python3 nor +# $RUNTIME, so it is safe this early. +[ -f "$ENV" ] || { [ -f .env.example ] && cp .env.example "$ENV" || touch "$ENV"; } + # ── prerequisites ──────────────────────────────────────────────────────────── # Checked here, after flag parsing (so --help still works on a bare machine) and before the first line # that needs either one — setenv() below is already python3. Both are hard requirements, so report every @@ -109,33 +118,92 @@ done # python3 every .env write and every JSON response in this script and in scripts/ is parsed with it. # No version floor: only the stdlib (json, sys, os, datetime, base64) is used, so anything # still called python3 is new enough. -# docker the whole stack, and every extension build, runs in containers. Compose must be v2, i.e. -# the `docker compose` subcommand — the standalone `docker-compose` v1 binary is not used. +# RUNTIME the whole stack, and every extension build, runs in containers. Which CLI drives them is +# resolved by scripts/_runtime.sh (RUNTIME in the environment or .env, else auto-detected); +# it must provide a v2-style `compose` subcommand. The standalone `docker-compose` v1 binary +# is not used. # -# The daemon-not-running case is a warning, not an error: `docker compose pull` further down reports it -# far better than we can, and failing here would block the flag-only paths that never touch the daemon. +# The daemon-not-running case is a warning, not an error: `$RUNTIME compose pull` further down reports +# it far better than we can, and failing here would block the flag-only paths that never touch it. MISSING="" command -v python3 >/dev/null 2>&1 || MISSING="${MISSING} • python3 — not on PATH. macOS: brew install python3 (or install Xcode command line tools). Debian/Ubuntu: sudo apt-get install -y python3. RHEL/Amazon Linux: sudo dnf install -y python3." -if ! command -v docker >/dev/null 2>&1; then - MISSING="${MISSING} - • docker — not on PATH. Install Docker Desktop (https://docs.docker.com/get-docker/), Colima, or - Rancher Desktop, then re-run." -elif ! docker compose version >/dev/null 2>&1; then - # `docker` exists but has no `compose` subcommand: either Compose v1 only, or a plugin-less install. + +# runtime_resolve must run in THIS shell, not a command substitution: it sets RUNTIME as a side effect, +# which a subshell would discard. So its diagnostics go straight to stderr rather than into $MISSING — +# the one-pass property is preserved by deferring the exit until python3 has been reported too. +RUNTIME_OK=1 +runtime_resolve || RUNTIME_OK=0 +if [ "$RUNTIME_OK" = 1 ] && ! "$RUNTIME" compose version >/dev/null 2>&1; then + # The CLI exists but has no `compose` subcommand: Compose v1 only, or a plugin-less install. For + # podman that usually means the podman-compose (or docker-compose) provider is missing, since podman + # shells out to one rather than implementing compose itself. MISSING="${MISSING} - • docker compose (v2) — 'docker compose version' failed. The standalone docker-compose v1 binary is - not enough; install the Compose v2 plugin, or upgrade Docker Desktop." + • $RUNTIME compose — '$RUNTIME compose version' failed. A v2-style compose subcommand is required; + the standalone docker-compose v1 binary is not enough. + docker — install the Compose v2 plugin, or upgrade Docker Desktop. + podman — it shells out to a compose provider rather than implementing compose itself. + Podman Desktop can install one for you, or: brew install podman-compose" + # Before reciting prerequisites, check whether another supported runtime is simply ready. Hit in + # practice with RUNTIME=podman pinned in .env, podman's socket broken, and docker up with compose v2: + # the advice above is all correct and all irrelevant when a working runtime is already installed. + _calt="$(runtime_alternatives_compose)" + _pinned="$(runtime_requested)" + for _c in $_calt; do + if [ -n "$_pinned" ]; then + MISSING="${MISSING} + $_c is installed, responding, and has compose. RUNTIME is currently pinned to '$_pinned' — that + pin outranks auto-detection, which would have chosen $_c. Change it in .env: + RUNTIME=$_c" + else + MISSING="${MISSING} + $_c is installed, responding, and has compose. Set it in .env: + echo 'RUNTIME=$_c' >> .env" + fi + MISSING="${MISSING} + Set it in .env rather than per-command: run.sh, the build scripts, logs.sh and stop.sh each + resolve the runtime independently, so a one-off on the command line leaves the next command on + the other runtime." + done + unset _calt _pinned _c fi if [ -n "$MISSING" ]; then echo "This kit needs a couple of things that aren't here yet:$MISSING" >&2 - exit 1 + RUNTIME_OK=0 +fi +[ "$RUNTIME_OK" = 1 ] || exit 1 +# If the resolved runtime is not answering, say so — and if another supported one IS, name it. Detection +# is presence-based, so a stopped Docker Desktop alongside a healthy podman resolves to docker and then +# fails at 'Pulling images' with nothing pointing at the way out. +if ! "$RUNTIME" info >/dev/null 2>&1; then + echo "Note: $RUNTIME doesn't look ready (daemon not running, or no connection) — sort that out before this gets to 'Pulling images'." >&2 + _alt="$(runtime_alternatives)" + if [ -n "$_alt" ]; then + for _a in $_alt; do + echo " $_a is installed and responding. To use it, set it in .env — every script reads it" >&2 + echo " there, and a one-off on the command line would leave the next command elsewhere:" >&2 + echo " echo 'RUNTIME=$_a' >> .env" >&2 + done + fi + unset _alt _a fi -docker info >/dev/null 2>&1 || echo "Note: the Docker daemon doesn't look like it's running — start it before this gets to 'Pulling images'." >&2 + +# Sized-VM check. Separate from the MISSING one-pass above because it needs RUNTIME already resolved, +# and it is a hard failure by design: an undersized podman machine does not stop the stack from coming +# up, it silently poisons every extension build later with an OOM kill that reports itself as a Go +# deadlock and 'exit status 137'. A no-op on docker and on a machine-less podman. +runtime_machine_check || exit 1 + +# Rosetta preflight. Same shape and the same reason as the sized-VM check above: on Apple Silicon a +# podman machine without Rosetta hands amd64 binaries to QEMU, which cannot run the studio's .NET +# runtime — and every layer of that failure lies. The container reports "Up", /healthz simply never +# answers, and the wait loop further down burns 4.5 minutes before exiting on "Login failed", which +# points at credentials. Catching it here turns all of that into one accurate message. A no-op on +# docker, on an amd64 host, and on an arm64 studio image. +runtime_rosetta_check || exit 1 # ── .env helpers (line-based; safe for tokens/keys with special chars) ──────── -[ -f "$ENV" ] || { [ -f .env.example ] && cp .env.example "$ENV" || touch "$ENV"; } getenv() { grep -E "^$1=" "$ENV" 2>/dev/null | head -1 | cut -d= -f2- || true; } setenv() { python3 - "$ENV" "$1" "${2-}" <<'PY' @@ -582,7 +650,7 @@ fi # whenever that fetch failed. --reset-license (below) is how you ask for it to actually go away. if [ "$RESET" = 1 ]; then echo "==> --reset: tearing down stack + volumes (DB, extensions, file store)…" - docker compose down -v 2>/dev/null || true + "$RUNTIME" compose down -v 2>/dev/null || true for k in Authentication__LocalAdminEmail Authentication__LocalAdminPassword Authentication__SuperUsers \ DEVKIT_MODEL ANTHROPIC_API_KEY AWS_REGION AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN \ CLAUDE_MODEL CLAUDE_EXTRA_MODELS "${GATEWAY_KEYS[@]}" "${SUBSCRIPTION_KEYS[@]}" \ @@ -868,7 +936,7 @@ if [ -z "$MODEL" ]; then fi # CONTAINER_IMDS is ok | blocked | unknown:. Only a definite `blocked` withholds option 3 — # that one means the container test ran and failed (the hop limit), so the role would break at - # runtime. `unknown:` means we couldn't run the test at all (no docker yet, daemon down, busybox + # runtime. `unknown:` means we couldn't run the test at all (no runtime yet, daemon down, busybox # not pullable); the role itself is proven, so offer it with the caveat rather than hiding a # working option because our own check couldn't execute. if [ "$BEDROCK_AVAILABLE" = 1 ] && [ "$CONTAINER_IMDS" != blocked ]; then @@ -997,8 +1065,11 @@ for v in STUDIO_TAG UI_TAG; do done # ── start the stack ─────────────────────────────────────────────────────────── -echo "==> Pulling images…"; docker compose pull -echo "==> Starting…"; docker compose up -d +# Name the runtime here, not only when something breaks: auto-detect is presence-based and docker-first, +# so on a box with both installed the choice is otherwise invisible until it goes wrong. +echo "==> Using $(runtime_label)" +echo "==> Pulling images…"; "$RUNTIME" compose pull +echo "==> Starting…"; "$RUNTIME" compose up -d STUDIO_PORT="$(getenv STUDIO_PORT)"; [ -z "$STUDIO_PORT" ] && STUDIO_PORT=60021 UI_PORT="$(getenv UI_PORT)"; [ -z "$UI_PORT" ] && UI_PORT=4200 diff --git a/scripts/_builder.sh b/scripts/_builder.sh index 06efb4d..aae8ea1 100644 --- a/scripts/_builder.sh +++ b/scripts/_builder.sh @@ -4,13 +4,21 @@ # # Why a container at all: the dev-kit's runtime is entirely containerized, but the build was not — it # hard-required `jq dotnet npm zip curl unzip` on the developer's laptop (and `jq` isn't shipped by -# macOS, so it was an undeclared prerequisite that failed at line 15). Docker Compose is now the only -# prerequisite. +# macOS, so it was an undeclared prerequisite that failed at line 15). A compose-capable container CLI +# is now the only prerequisite. +# +# Every container command here goes through $RUNTIME (see scripts/_runtime.sh, sourced below) rather +# than a literal `docker`, so the same build works under docker or podman. # # REQUIRES scripts/_target.sh, sourced FIRST: this file uses its `_envv` (to read .env for BUILDER_IMAGE / # BUILDER_PULL / BUILDER_TAG / STUDIO_PORT) and its `_TARGET`. Both current callers do source it first; # that is a requirement of sourcing this file, not an accident of their layout. +# BASH_SOURCE, not $0: $0 is whichever script sourced this one, and build-extension.sh may be invoked +# from any cwd. _runtime.sh is idempotent, so sourcing it here as well as in run.sh costs nothing. +# shellcheck source=scripts/_runtime.sh +. "${BASH_SOURCE[0]%/*}/_runtime.sh" + # builder_truthy -> echoes 1 or 0; returns 1 (no output) if isn't a boolean at all. # # Accepts the spellings a human actually types into an env var, case-insensitively, because @@ -26,26 +34,26 @@ builder_truthy() { esac } -# builder_mode -> native|container|error: +# builder_mode -> native|container|error: # # Kept as a pure function of four booleans (no `command -v`, no filesystem probes) so the whole -# decision matrix can be exercised without Docker, a platform, or a toolchain. +# decision matrix can be exercised without a container runtime, a platform, or a toolchain. builder_mode() { # Normalize every argument up front, before any branch is taken, so a bad value can't be silently # read as false by whichever comparison happens to see it first. Separate declaration from assignment: # `local x=$(cmd)` masks the command's exit status behind `local`'s own, so the || would never fire. - local force_native in_container have_toolchain have_docker + local force_native in_container have_toolchain have_runtime force_native=$(builder_truthy "${1-}") || { echo error:badarg; return; } in_container=$(builder_truthy "${2-}") || { echo error:badarg; return; } have_toolchain=$(builder_truthy "${3-}") || { echo error:badarg; return; } - have_docker=$(builder_truthy "${4-}") || { echo error:badarg; return; } + have_runtime=$(builder_truthy "${4-}") || { echo error:badarg; return; } # The explicit escape hatch outranks everything: --native / DUPLO_BUILD_NATIVE=1 means the caller # has their own toolchain and wants it used, so don't second-guess them. [ "$force_native" = 1 ] && { echo native; return; } - # Already inside a container. Nesting is not an option — the agent container has the docker CLI but NO - # /var/run/docker.sock, so builder_probe_docker's daemon check fails — and it isn't needed either, + # Already inside a container. Nesting is not an option — the agent container has a container CLI but + # no socket to talk to, so builder_probe_runtime's reachability check fails — and it isn't needed either, # because the agent image ships the whole toolchain. This is the branch the agent's build takes. It is # also the branch the re-launched build takes inside the builder image, which stops the wrapper recursing. if [ "$in_container" = 1 ]; then @@ -53,14 +61,14 @@ builder_mode() { echo error:toolchain; return fi - # A host with Docker: containerize, even if a local toolchain exists, so that laptop, CI and agent - # builds all use one pinned toolchain. `--native` is there for anyone who wants their own. - [ "$have_docker" = 1 ] && { echo container; return; } + # A host with a working runtime: containerize, even if a local toolchain exists, so that laptop, CI + # and agent builds all use one pinned toolchain. `--native` is there for anyone who wants their own. + [ "$have_runtime" = 1 ] && { echo container; return; } - # No Docker. A complete local toolchain still works, so degrade gracefully rather than fail (this - # keeps a Docker-less CI runner building). + # No runtime. A complete local toolchain still works, so degrade gracefully rather than fail (this + # keeps a runtime-less CI runner building). [ "$have_toolchain" = 1 ] && { echo native; return; } - echo error:docker + echo error:runtime } # The tools build-extension.sh has always required. One list, used both to decide (builder_probe_toolchain) @@ -96,21 +104,98 @@ builder_probe_toolchain() { echo 1 } -builder_probe_docker() { - # The CLI existing is not enough — the agent image HAS the docker binary and no daemon, so - # `command -v docker` alone would answer 1 there and send the build looking for a socket that isn't - # mounted. Only a reachable daemon counts. +builder_probe_runtime() { + # The CLI existing is not enough — the agent image HAS a client binary and nothing to talk to, so + # `command -v` alone would answer 1 there and send the build looking for a socket that isn't mounted. + # Only a runtime that actually answers counts. + # + # `version --format` rather than `info`: rendering any part of the SERVER block requires contacting + # the server, so it proves exactly as much, and it is ~25x cheaper (measured 0.042s vs 1.074s). + # builder_dispatch evaluates all four probes as *arguments* to builder_mode, so bash runs this one + # even when --native has already decided the answer — and build-all.sh --native pays it once per + # extension. Podman is daemonless, but it implements the same query (reporting its own version as + # the server), so one probe covers every supported runtime. + # + # `{{json .Server}}` and NOT `{{.Server.Version}}`: the server struct is not the same shape across + # the supported CLIs. docker and podman put a version string at .Server.Version; finch (nerdctl + # underneath) does not have that field at all — its server block nests the component versions + # instead, so the narrower template dies in the template engine rather than on the wire: + # + # finch version --format '{{.Server.Version}}' + # fatal: template: version:1:9: executing "version" at <.Server.Version>: + # can't evaluate field Version in type main.NerdctlServerOutput # - # `version --format {{.Server.Version}}` rather than `info`: printing the SERVER version requires - # contacting the server, so it proves exactly as much, and it is ~25x cheaper (measured 0.042s vs - # 1.074s). builder_dispatch evaluates all four probes as *arguments* to builder_mode, so bash runs - # this one even when --native has already decided the answer — and build-all.sh --native pays it once - # per extension. - if command -v docker >/dev/null 2>&1 && docker version --format '{{.Server.Version}}' >/dev/null 2>&1; then - echo 1 + # That is indistinguishable from an unreachable runtime here, so a perfectly healthy finch was + # reported as "no container runtime is available" and the build refused to run. Asking for the whole + # object renders on all four and still fails when nothing answers — a stopped podman machine exits + # 125 (printing `null`) and a stopped Docker Desktop errors the same way, so the probe stays honest. + [ -n "${RUNTIME:-}" ] || { echo 0; return; } + command -v "$RUNTIME" >/dev/null 2>&1 || { echo 0; return; } + if "$RUNTIME" version --format '{{json .Server}}' >/dev/null 2>&1; then echo 1; else echo 0; fi +} + +# builder_runtime_message +# +# Formats the "this build cannot be containerized" failure. Pure — it probes nothing — so every branch +# below is exercised in the tests on any machine. +# +# One message used to cover three situations and got two of them wrong. It said "Install docker or +# podman" to anyone who had merely not started Docker Desktop, which is the most common way to arrive +# here, and to anyone running a perfectly healthy podman that detection had passed over. What is +# actually true differs per case, so the text does too: +# +# nothing on PATH -> installing really is the fix +# present but not answering -> start it +# ...and another one answers -> use that one, with the command that does it +builder_runtime_message() { + local rt="${1-}" alts="${2-}" missing="${3-}" requested="${4-0}" a + + if [ -z "$rt" ]; then + echo "ERROR: no container runtime found on PATH, and the local toolchain is incomplete." >&2 + echo " Missing: ${missing:-(none)}" >&2 + echo " Install docker or podman (each needs a v2-style 'compose' subcommand) — that is the" >&2 + echo " only prerequisite — or install the toolchain yourself and re-run with --native." >&2 + return + fi + + if [ "$requested" = 1 ]; then + echo "ERROR: '$rt' was requested (RUNTIME) but is not responding, so this build cannot be" >&2 + echo " containerized." >&2 else - echo 0 + echo "ERROR: '$rt' was auto-detected but is not responding, so this build cannot be containerized." >&2 + echo " Detection tests whether a CLI is on PATH, not whether it is running." >&2 fi + + if [ -n "$alts" ]; then + echo "" >&2 + # shellcheck disable=SC2086 # deliberate word-split: alts is a space-separated list + for a in $alts; do + if [ "$requested" = 1 ]; then + echo " $a is installed and responding. RUNTIME is pinned to '$rt' — change it in .env:" >&2 + else + echo " $a is installed and responding. Set it in .env:" >&2 + fi + echo "" >&2 + echo " RUNTIME=$a" >&2 + echo "" >&2 + echo " In .env and not on the command line: run.sh, the build scripts, logs.sh and stop.sh" >&2 + echo " each resolve the runtime independently, so a one-off would leave the next command on" >&2 + echo " the other runtime — which is how a build ends up unable to see the running studio." >&2 + done + echo "" >&2 + echo " ...or start $rt and re-run." >&2 + else + echo "" >&2 + case "$rt" in + docker) echo " Start it and re-run — on macOS, launch Docker Desktop and wait for it to" >&2 + echo " report Running." >&2 ;; + podman) echo " Start it and re-run: podman machine start" >&2 ;; + *) echo " Start it and re-run." >&2 ;; + esac + fi + + echo "" >&2 + echo " Or build with a toolchain you installed yourself: --native (missing now: ${missing:-none})" >&2 } builder_missing_tools() { @@ -124,8 +209,8 @@ builder_missing_tools() { # --- Image resolution ---------------------------------------------------------------------------- # _envv is a raw `grep | cut`, so whatever follows the `=` in .env survives verbatim: a CRLF-edited file -# yields "v1", a quoted value yields "\"v1\"", trailing spaces stay spaces. Docker rejects all three -# with "invalid reference format". Strip the three that are unambiguously accidental rather than fail on +# yields "v1", a quoted value yields "\"v1\"", trailing spaces stay spaces. Every runtime rejects all +# three with "invalid reference format". Strip the three that are unambiguously accidental rather than fail on # them — nobody means to pin a tag whose name contains a carriage return. builder_clean_value() { local v="${1-}" @@ -137,7 +222,7 @@ builder_clean_value() { printf '%s' "$v" } -# Resolve BUILDER_IMAGE, pulling (or, if that fails, building locally) so `docker compose run` finds it. +# Resolve BUILDER_IMAGE, pulling (or, if that fails, building locally) so `$RUNTIME compose run` finds it. # A local fallback keeps the dev-kit usable before the image is published, and for anyone without quay # access. Every success path EXPORTS the name: compose reads it from the environment, and an unexported # one means compose silently falls back to the `${BUILDER_IMAGE:-quay…}` default in docker-compose.yml — @@ -146,7 +231,7 @@ builder_clean_value() { # Caching: an image that is already local is used as-is, without contacting the registry, so a warm build # never pays for a pull. The cost is that a MOVING tag is never refreshed — and BUILDER_TAG=latest, our # default, is exactly that, on an image whose content is a .NET SDK. BUILDER_PULL=1 forces the re-pull; -# without it the only recourse would be `docker rmi`. +# without it the only recourse would be `$RUNTIME rmi`. # # Both knobs are read from the environment AND from .env (via _envv), because compose interpolates # docker-compose.yml against .env too: if .env's BUILDER_IMAGE were ignored here, compose would honour it @@ -155,9 +240,9 @@ builder_clean_value() { # error — "offline with BUILDER_PULL=1 set" is the ordinary way to get here, and a stale toolchain beats no # build. Returns 0 when BUILDER_IMAGE is usable as-is. builder_keep_local_copy() { # builder_keep_local_copy - docker image inspect "$BUILDER_IMAGE" >/dev/null 2>&1 || return 1 + "$RUNTIME" image inspect "$BUILDER_IMAGE" >/dev/null 2>&1 || return 1 echo "WARNING: could not re-pull $BUILDER_IMAGE, using the local copy." >&2 - echo " docker pull said: $(printf '%s' "${1-}" | tail -1)" >&2 + echo " $RUNTIME pull said: $(printf '%s' "${1-}" | tail -1)" >&2 } builder_resolve_image() { @@ -174,18 +259,18 @@ builder_resolve_image() { img="$(builder_clean_value "${BUILDER_IMAGE:-$(_envv BUILDER_IMAGE)}")" if [ -n "$img" ]; then BUILDER_IMAGE="$img"; export BUILDER_IMAGE - if [ "$force_pull" != 1 ] && docker image inspect "$BUILDER_IMAGE" >/dev/null 2>&1; then return 0; fi - pull_err="$(docker pull -q "$BUILDER_IMAGE" 2>&1)" && return 0 + if [ "$force_pull" != 1 ] && "$RUNTIME" image inspect "$BUILDER_IMAGE" >/dev/null 2>&1; then return 0; fi + pull_err="$("$RUNTIME" pull -q "$BUILDER_IMAGE" 2>&1)" && return 0 builder_keep_local_copy "$pull_err" && return 0 # A pinned image that cannot be obtained is fatal: the caller named one specific image, so quietly # building a different one would not be doing what they asked. echo "ERROR: BUILDER_IMAGE=$BUILDER_IMAGE is neither present locally nor pullable." >&2 - echo " docker pull said: $(printf '%s' "$pull_err" | tail -1)" >&2 + echo " $RUNTIME pull said: $(printf '%s' "$pull_err" | tail -1)" >&2 return 1 fi local tag; tag="$(builder_clean_value "$(_envv BUILDER_TAG)")"; tag="${tag:-latest}" - # Docker's own tag grammar: [A-Za-z0-9_][A-Za-z0-9._-]{0,127}. Checked here because the alternative is + # OCI tag grammar: [A-Za-z0-9_][A-Za-z0-9._-]{0,127}. Checked here because the alternative is # a pull that fails for an unobvious reason and a silent fall-through to building something else. case "$tag" in *[!A-Za-z0-9._-]* | [!A-Za-z0-9_]* ) @@ -195,24 +280,24 @@ builder_resolve_image() { return 1 ;; esac BUILDER_IMAGE="quay.io/duplocloud/duplo-extension-builder:$tag"; export BUILDER_IMAGE - if [ "$force_pull" != 1 ] && docker image inspect "$BUILDER_IMAGE" >/dev/null 2>&1; then return 0; fi + if [ "$force_pull" != 1 ] && "$RUNTIME" image inspect "$BUILDER_IMAGE" >/dev/null 2>&1; then return 0; fi echo "==> Fetching the build toolchain image ($BUILDER_IMAGE)" >&2 - pull_err="$(docker pull -q "$BUILDER_IMAGE" 2>&1)" && return 0 + pull_err="$("$RUNTIME" pull -q "$BUILDER_IMAGE" 2>&1)" && return 0 builder_keep_local_copy "$pull_err" && return 0 # Not published, or no quay access — build it from the Dockerfile that ships in this repo. `build` is # the context, not `.`: the Dockerfile COPYs nothing from the context, so sending the whole repo - # (samples/, packages/, node_modules/) to the daemon would be pure waste. + # (samples/, packages/, node_modules/) to the builder would be pure waste. # # The pull error is reported, not swallowed: "unauthorized", "manifest unknown" and a registry rate # limit all land here, and they are the difference between "the image isn't published yet" (fine, build # it) and "your credentials expired" (which a two-minute local build silently papers over). echo " pull failed: $(printf '%s' "$pull_err" | tail -1)" >&2 BUILDER_IMAGE="duplo-extension-builder:local"; export BUILDER_IMAGE - docker image inspect "$BUILDER_IMAGE" >/dev/null 2>&1 && return 0 + "$RUNTIME" image inspect "$BUILDER_IMAGE" >/dev/null 2>&1 && return 0 echo " building it locally from build/Dockerfile.builder instead (one time, ~2 min)" >&2 - docker build -q -f build/Dockerfile.builder -t "$BUILDER_IMAGE" build >/dev/null \ + "$RUNTIME" build -q -f build/Dockerfile.builder -t "$BUILDER_IMAGE" build >/dev/null \ || { echo "ERROR: could not build the builder image from build/Dockerfile.builder" >&2; return 1; } } @@ -220,17 +305,18 @@ builder_resolve_image() { # builder_studio_state -> in-project | absent | unknown # -# One `docker compose ps` (~0.06s isolated, ~0.11s amortized against a build) — this sits in front of -# every containerized build, so it has to stay a single call. +# One runtime_compose_running_services call (~0.06s isolated, ~0.11s amortized against a build) — this +# sits in front of every containerized build, so it has to stay a single call. That helper is also what +# keeps the answer correct on podman, whose `compose ps` has no --status/--services flags. # -# `docker compose` is PROJECT-scoped, and the project defaults to the directory name. So this answers a +# Compose is PROJECT-scoped, and the project defaults to the directory name. So this answers a # narrower question than "is the platform up": it answers "is the studio on the network this build will # join". A second checkout (or a git worktree, which is how this repo is developed) is a different # project, and its running studio is invisible here — hence three states, not two: `absent` is evidence, # `unknown` (the query itself failed) is not, and they must not lead to the same message. builder_studio_state() { local running - running="$(docker compose ps --status running --services 2>/dev/null)" || { echo unknown; return; } + running="$(runtime_compose_running_services)" || { echo unknown; return; } if printf '%s\n' "$running" | grep -qx duplo-ai-studio; then echo in-project; else echo absent; fi } @@ -257,7 +343,7 @@ builder_studio_url() { # success rather than a guess, so this says what it is doing instead of refusing to try. `unknown` # stays silent: a scary message in front of every build, on no evidence, teaches people to ignore it. if [ "$state" = absent ]; then - echo "WARNING: no running duplo-ai-studio in this compose project (docker compose scopes services" >&2 + echo "WARNING: no running duplo-ai-studio in this compose project ($RUNTIME compose scopes services" >&2 echo " per project, and this one is the directory '$(basename "$PWD")'), so the build will" >&2 echo " try the published host port instead: $url" >&2 echo " That reaches a platform started from another checkout, or by other means. If nothing" >&2 @@ -266,6 +352,71 @@ builder_studio_url() { printf '%s' "$url" } +# builder_run_direct [args...] +# +# The finch path: run the builder image directly rather than through `compose run`. Its compose +# implementation cannot express this invocation at all — three independent reasons, each verified against +# finch v1.19.0 (nerdctl v2.2.2 underneath), and the first two fail SILENTLY: +# +# 1. INTERPOLATION IGNORES THE SHELL ENVIRONMENT. It reads ${VAR} from the .env file only, so every value +# builder_dispatch exports falls through to its `:-` default instead. With DUPLO_BASE, DUPLO_UID/GID +# and BUILDER_IMAGE all exported, `finch compose config` still rendered `DUPLO_BASE: ""`, +# `user: 1000:1000` and the default image. An empty DUPLO_BASE then resolves in _target.sh to +# http://localhost:, which inside the container is the CONTAINER — so the SDK fetch dies +# on a URL that looks right; and uid 1000 writes the bundle as an id the caller may not own. +# 2. NO WAY TO SET THE IMAGE. `compose run` takes it from the compose file and has no --image flag, so +# together with (1) a pinned BUILDER_TAG/BUILDER_IMAGE has no route through at all. +# 3. A TTY IS FORCED. It always allocates one, and every spelling of the opt-out (--tty=false, -t, -T, +# --no-TTY) is rejected — so a caller whose stdout is not a console (CI, a script, an agent) dies on +# "provided file is not a console" before the build starts. +# +# `compose --env-file` would have covered 1 and 2 together; finch lists it in `compose --help` but +# mistranslates it into a `-e` that nerdctl rejects underneath, so it is not a way out either. +# +# EVERY FLAG BELOW MIRRORS THE `builder` SERVICE IN docker-compose.yml. That duplication is the price of +# this path — change the service and change this with it. +builder_run_direct() { + local script="$1"; shift + + # Seeded non-empty on purpose: macOS ships bash 3.2, where expanding an EMPTY array under `set -u` + # ("${args[@]}") is an unbound-variable error. + local args + args=(run --rm -w /work -u "$DUPLO_UID:$DUPLO_GID" -v "$PWD:/work") + + # The per-uid package caches, named exactly as the top-level `volumes:` block names them, so this path + # and the compose path warm the SAME caches rather than each paying for a cold build. + args+=(-v "duplo_devkit_nuget_$DUPLO_UID:/cache/nuget") + args+=(-v "duplo_devkit_npm_$DUPLO_UID:/cache/npm") + + # extra_hosts: the route to a studio that is not on this compose network. + args+=(--add-host "host.docker.internal:host-gateway") + + # Join the project network ONLY when the studio is in this project — that is exactly the case in which + # builder_studio_url chose the bare service name http://duplo-ai-studio:60021, which nothing off that + # network can resolve. When it is absent or unknown the URL is the published host port, which is + # reachable without joining anything (and the network may not even exist). + if [ "$(builder_studio_state)" = in-project ]; then + args+=(--net "$(runtime_compose_project)_default") + fi + + # platform: ${BUILDER_PLATFORM:-} — omitted entirely when unset, so the build stays on the host's + # native architecture instead of silently emulating one. + if [ -n "${BUILDER_PLATFORM:-}" ]; then + args+=(--platform "$BUILDER_PLATFORM") + fi + + # The environment: block, value for value. Passed as real flags, which is the whole point: unlike + # compose interpolation above, `run -e` does not consult .env and cannot silently drop them. + args+=(-e "DUPLO_BUILD_NATIVE=1") + args+=(-e "DUPLO_TARGET=${DUPLO_TARGET:-local}") + args+=(-e "DUPLO_BASE=${DUPLO_BASE:-}") + args+=(-e "DUPLO_HOST=${DUPLO_HOST:-}") + args+=(-e "DUPLO_TOKEN=${DUPLO_TOKEN:-}") + args+=(-e "DUPLO_ADMIN_TOKEN=${DUPLO_ADMIN_TOKEN:-}") + + exec "$RUNTIME" "${args[@]}" "$BUILDER_IMAGE" "$script" "$@" +} + # builder_dispatch [args...] # Returns 0 to mean "carry on natively". Otherwise it runs the build in the container and exits with # the container's status — it never returns in that case. @@ -287,25 +438,33 @@ builder_dispatch() { # value we actually resolved, which is not necessarily the one in the environment. local want_native; want_native="$(builder_clean_value "${DUPLO_BUILD_NATIVE:-$(_envv DUPLO_BUILD_NATIVE)}")" + # Resolve the CLI before probing. Failure is only fatal when a runtime was NAMED: the caller said + # which one to use, and quietly building with a different one — or dropping to a native build they + # did not ask for — is not what they asked for. Auto-detect finding nothing stays silent, because + # builder_mode's native fallback below still builds on a machine with the full toolchain, and that is + # what keeps a runtime-less CI runner working. The re-run is to print the specific reason, which was + # suppressed on the first call. + runtime_resolve 2>/dev/null || { + if [ -n "$(runtime_requested)" ]; then runtime_resolve; exit 1; fi + } + local mode; mode="$(builder_mode \ "$want_native" \ "$(builder_probe_in_container)" \ "$(builder_probe_toolchain)" \ - "$(builder_probe_docker)")" + "$(builder_probe_runtime)")" case "$mode" in native) return 0 ;; error:toolchain) echo "ERROR: missing build tool(s): $(builder_missing_tools)" >&2 - echo " This is running inside a container with no Docker daemon, so the build cannot be" >&2 + echo " This is running inside a container with no reachable runtime, so the build cannot be" >&2 echo " delegated. Use an image that ships the toolchain (.NET SDK 8 + Node 22), e.g." >&2 echo " build/Dockerfile.builder in this repo." >&2 exit 1 ;; - error:docker) - echo "ERROR: Docker is not available and the local toolchain is incomplete." >&2 - echo " Missing: $(builder_missing_tools)" >&2 - echo " Install Docker (Compose v2) — that is the only prerequisite — or install the" >&2 - echo " toolchain yourself and re-run with --native." >&2 + error:runtime) + builder_runtime_message "${RUNTIME:-}" "$(runtime_alternatives)" "$(builder_missing_tools)" \ + "$([ -n "$(runtime_requested)" ] && echo 1 || echo 0)" exit 1 ;; container) ;; # error:badarg, or a mode this function hasn't been taught. The only way to reach it in practice is @@ -348,13 +507,35 @@ builder_dispatch() { builder_resolve_image || exit 1 - # Run as the caller so dist/ is theirs, not root's — BOTH halves: uid alone leaves the group as the - # compose default, i.e. a bundle the caller's group cannot write. - DUPLO_UID="$(id -u)" DUPLO_GID="$(id -g)" + # Run as whichever ids make dist/ come out owned by the caller rather than by root or by an + # unreachable subuid — BOTH halves: uid alone leaves the group as the compose default, i.e. a bundle + # the caller's group cannot write. Which ids those are differs per runtime, so the choice lives in + # runtime_builder_ids (see the long note there: rootless podman needs 0:0, docker needs the real uid). + read -r DUPLO_UID DUPLO_GID <<<"$(runtime_builder_ids)" export DUPLO_UID DUPLO_GID BUILDER_IMAGE DUPLO_BASE DUPLO_TARGET="${_TARGET:-local}" - echo "==> Building in $BUILDER_IMAGE (uid $DUPLO_UID:$DUPLO_GID) — --native to use your own toolchain" + echo "==> Building in $BUILDER_IMAGE via $(runtime_label) (uid $DUPLO_UID:$DUPLO_GID)" \ + "— --native to use your own toolchain" # --rm: the container is a one-shot. --no-deps: never start the platform as a side effect of a build. - # -T: nothing in a build wants a TTY, and this is exec'd from a script whose stdio may already be a pipe. - exec docker compose run --rm -T --no-deps builder "$script" "$@" + # + # --profile tools is REQUIRED, not belt-and-braces: the builder service sits behind that profile, and + # while `docker compose run` implicitly enables the profiles of the service it was asked to run, + # podman-compose 1.5 does not — it reports "missing services [builder]" and exits 1. Naming the + # profile explicitly is a no-op on docker and the only way in on podman. (COMPOSE_PROFILES=tools is + # NOT an alternative here: podman-compose ignores that variable.) + # + # -T (don't allocate a pseudo-TTY) is what makes this safe to exec from a script whose stdio is + # already a pipe — CI, or an agent. Nothing in a build wants a TTY. + # + # TWO DISPATCH PATHS, because `compose run` is only usable on half the supported CLIs. docker and + # podman keep the original invocation verbatim. finch cannot use compose for this at all: it rejects + # -T outright ("unknown shorthand flag: 'T' in -T") and then forces a TTY anyway with no opt-out, on + # top of ignoring the exported DUPLO_BASE/DUPLO_UID/BUILDER_IMAGE entirely — so it runs the image + # directly instead. builder_run_direct has the evidence for each of those. + case "$RUNTIME" in + docker|podman) + exec "$RUNTIME" compose --profile tools run --rm -T --no-deps builder "$script" "$@" ;; + *) + builder_run_direct "$script" "$@" ;; + esac } diff --git a/scripts/_runtime.sh b/scripts/_runtime.sh new file mode 100644 index 0000000..2b33521 --- /dev/null +++ b/scripts/_runtime.sh @@ -0,0 +1,671 @@ +# shellcheck shell=bash +# Sourced FIRST by run.sh, stop.sh, logs.sh and scripts/* — resolves the container CLI once, so every +# call site reads `$RUNTIME run` / `$RUNTIME compose` instead of hard-coding `docker`. +# +# Why a variable rather than a `docker` shim on PATH: a shim is per-machine, invisible in the repo, and +# silently changes what every other tool on the box does too. RUNTIME is per-checkout, lives in .env +# next to every other knob (BUILDER_TAG, DUPLO_TARGET…), and is visible in the one place a reader of +# these scripts already looks. +# +# WHAT COUNTS AS A RUNTIME HERE +# Every value below is a *client CLI* that speaks Docker's command grammar — `run`, `pull`, `build`, +# `info`, `version`, `image inspect` — AND ships a `compose` subcommand, because this kit drives the +# whole stack through Compose. That is the entire contract; anything meeting it can be added to +# _RUNTIME_SUPPORTED — but check the dispatch split in scripts/_builder.sh first, because `compose run` +# is not usable on every CLI that meets the contract, and the fallback branch there assumes a +# finch-shaped CLI. +# +# docker the default, and what the kit was written against. +# podman daemonless; rootless by default. See the uid note on runtime_builder_ids below. +# +# finch is TEMPORARILY de-listed, pending validation — it is not auto-detected and RUNTIME=finch is +# rejected. This is a claim-reduction, not a removal: every finch code path is still here and still +# correct, including the compose-bypass dispatch in scripts/_builder.sh and its notes on the three +# ways finch's `compose` fails silently. Re-enable by putting `finch` back in the array below; nothing +# else needs touching. The reason for de-listing is the same one that removed nerdctl: the kit should +# not advertise a runtime nobody is currently exercising. +# +# nerdctl is deliberately NOT here, and is not coming back. finch is the supported way to drive +# containerd in this kit: it is nerdctl plus the VM and the defaults that make it work on macOS and +# Windows, which is where anyone here is running containerd in the first place. Bare nerdctl was listed +# once and never exercised, and it shares finch's compose limitations without finch's setup. A +# native-Linux containerd user can still pin RUNTIME to a CLI of their choice by adding it back here — +# see the note above about the builder's dispatch split. +# +# runc is deliberately NOT here, though it was asked for. It is a *low-level OCI runtime*: it executes +# an already-unpacked bundle (a rootfs plus config.json) given by path, and has no concept of an image, +# a registry, a network or a compose file — `runc run busybox:1.36` cannot work, because runc never +# resolves an image name. It is the layer the CLIs above call *underneath* (this machine's podman drives +# `crun`, runc's C rewrite, via `--runtime`). Choosing the low-level runtime is a different knob from +# choosing the client, so runtime_resolve rejects it with that explanation rather than emitting commands +# that would fail later with something unrecognisable. + +# The order here is also the auto-detect precedence: docker first, so a machine that has always had +# Docker keeps behaving exactly as it did before this file existed. +# +# NOTE: precedence is by PRESENCE, not by liveness (see runtime_detect) — so an installed-but-stopped +# docker still wins over a running podman, and the build then fails rather than falling through. Pin +# RUNTIME in .env on a machine that has both. Installing Docker alongside an existing podman is the +# common way to land on the wrong one without noticing. +_RUNTIME_SUPPORTED=(docker podman) + +# Self-contained .env read. _target.sh defines an identical `_envv`, but this file is sourced by run.sh +# and stop.sh too, which never source _target.sh — and it must work before either has run. Same +# DUPLO_ENV_FILE convention, same `|| true` so a missing key yields empty instead of tripping the +# callers' `set -euo pipefail`. +_RUNTIME_ENV_FILE="${DUPLO_ENV_FILE:-.env}" +_runtime_envv() { grep -E "^$1=" "$_RUNTIME_ENV_FILE" 2>/dev/null | head -1 | cut -d= -f2- || true; } + +# Strip the three accidents a hand- or Windows-edited .env leaves behind. Same three cases, and the same +# reasoning, as builder_clean_value in _builder.sh — duplicated rather than shared because that file is +# sourced only by the two build scripts, and this one has to stand alone. +_runtime_clean() { + local v="${1-}" + v="${v//$'\r'/}" # CRLF line endings + case "$v" in \"*\") v="${v#\"}"; v="${v%\"}" ;; esac # "quoted" + case "$v" in \'*\') v="${v#\'}"; v="${v%\'}" ;; esac # 'quoted' + v="${v#"${v%%[![:space:]]*}"}" # leading whitespace + v="${v%"${v##*[![:space:]]}"}" # trailing whitespace + printf '%s' "$v" +} + +# runtime_detect -> first supported CLI on PATH, or empty. +# +# Presence only — no daemon check. Whether the thing actually answers is a separate question with a +# separate message (run.sh's preflight, builder_probe_runtime), and conflating them here would make an +# unstarted Docker Desktop silently fall through to podman. +runtime_detect() { + local r + for r in "${_RUNTIME_SUPPORTED[@]}"; do + command -v "$r" >/dev/null 2>&1 && { printf '%s' "$r"; return; } + done + printf '' +} + +# runtime_requested -> the explicitly requested runtime (environment, else .env), or empty when the +# choice is being left to auto-detection. +# +# The distinction matters to the build path: "you asked for X and X is unusable" is a hard error, +# because silently building with something else is not what was asked for — whereas "nothing found" +# can legitimately fall back to a native toolchain build. +# _runtime_answers -> 0 when that CLI is present AND something answers on the other end. +# +# Split out from runtime_alternatives for the reason runtime_rosetta_active is split from the Rosetta +# policy: the probe needs a real runtime, and the policy consuming it has to be testable without one. +# Same `version --format '{{json .Server}}'` form as builder_probe_runtime, and for the same reason — +# rendering any part of the SERVER block requires contacting the server, while the whole-object form +# still renders on a CLI whose server block is shaped differently (finch) instead of dying in the +# template engine and reading as "unreachable". +_runtime_answers() { + command -v "$1" >/dev/null 2>&1 || return 1 + "$1" version --format '{{json .Server}}' >/dev/null 2>&1 +} + +# runtime_alternatives -> the supported runtimes OTHER than the resolved one that actually answer. +# +# Exists so a failure can name a way out instead of reciting prerequisites. Detection is presence-based +# and docker-first (deliberately: a daemon check on every run is not worth the second), so a box with +# Docker Desktop installed-but-stopped alongside a healthy podman resolves to docker and then cannot +# build. The honest advice there is "use podman", which requires knowing podman answers. +runtime_alternatives() { + local r out="" + for r in "${_RUNTIME_SUPPORTED[@]}"; do + [ "$r" = "${RUNTIME:-}" ] && continue + _runtime_answers "$r" && out="$out $r" + done + printf '%s' "${out# }" +} + +# _runtime_has_compose -> 0 when that CLI has a v2-style `compose` subcommand. +# +# Its own function, like _runtime_answers, so the policy below is testable without either runtime. +_runtime_has_compose() { + "$1" compose version >/dev/null 2>&1 +} + +# runtime_alternatives_compose -> supported runtimes OTHER than the resolved one that answer AND have +# compose. +# +# Narrower than runtime_alternatives on purpose: a runtime can be reachable and still have no compose +# subcommand, and naming that one as the way out of a compose failure sends the person in a circle. +runtime_alternatives_compose() { + local r out="" + for r in "${_RUNTIME_SUPPORTED[@]}"; do + [ "$r" = "${RUNTIME:-}" ] && continue + _runtime_answers "$r" || continue + _runtime_has_compose "$r" || continue + out="$out $r" + done + printf '%s' "${out# }" +} + +runtime_requested() { _runtime_clean "${RUNTIME:-$(_runtime_envv RUNTIME)}"; } + +# runtime_resolve -> sets and exports RUNTIME; returns 1 with a message on stderr if it cannot. +# +# Precedence: environment → .env → auto-detect. Exported because the build path hands the choice down +# to a nested `$RUNTIME compose run` invocation of these same scripts, and an unexported RUNTIME there +# would silently re-detect and could pick a different CLI than the one the caller asked for. +# +# Idempotent: sourcing this file twice (build-extension.sh sources _builder.sh, which sources this) must +# not re-run detection or re-print anything. +runtime_resolve() { + [ -n "${_RUNTIME_RESOLVED:-}" ] && return 0 + + local want; want="$(_runtime_clean "${RUNTIME:-$(_runtime_envv RUNTIME)}")" + + if [ -z "$want" ]; then + want="$(runtime_detect)" + if [ -z "$want" ]; then + echo "ERROR: no container runtime found on PATH." >&2 + echo " This kit needs one of: ${_RUNTIME_SUPPORTED[*]} — each with a 'compose' subcommand." >&2 + echo " Install one, or set RUNTIME= in .env if yours is under a different name." >&2 + return 1 + fi + fi + + # runc and its drop-in siblings get their own message: "unsupported runtime: runc" would read as an + # oversight, and the person would reasonably try to fix it by installing runc — which is very likely + # already installed, and still will not work. + case "$want" in + runc|crun|youki|gvisor|runsc|kata-runtime) + echo "ERROR: '$want' is a low-level OCI runtime, not a container CLI, so it cannot drive this kit." >&2 + echo " It runs an unpacked bundle handed to it by path — it has no images, registries," >&2 + echo " networks or compose, so there is no '$want run ' or '$want compose up'." >&2 + echo " It is what the CLI below uses underneath; to select it, tell that CLI:" >&2 + echo " RUNTIME=podman and podman --runtime $want … (or docker's default-runtime)" >&2 + echo " Set RUNTIME to one of: ${_RUNTIME_SUPPORTED[*]}" >&2 + return 1 ;; + esac + + local r ok=0 + for r in "${_RUNTIME_SUPPORTED[@]}"; do [ "$want" = "$r" ] && { ok=1; break; }; done + if [ "$ok" != 1 ]; then + echo "ERROR: RUNTIME='$want' is not a supported container CLI." >&2 + echo " Supported: ${_RUNTIME_SUPPORTED[*]}" >&2 + echo " (A runtime qualifies if it speaks Docker's CLI grammar and has a 'compose' subcommand.)" >&2 + return 1 + fi + + if ! command -v "$want" >/dev/null 2>&1; then + echo "ERROR: RUNTIME='$want' was requested but '$want' is not on PATH." >&2 + return 1 + fi + + RUNTIME="$want"; export RUNTIME + _RUNTIME_RESOLVED=1 + + # podman routes `podman compose` to an external provider (podman-compose or docker-compose) and prints + # a four-line banner about having done so, to stderr, on EVERY call. This kit makes many compose calls + # per run, and the banner is noise the user cannot act on. Only set the opt-out when it is unset, so + # anyone who genuinely wants the banner can keep it. + [ "$RUNTIME" = podman ] && : "${PODMAN_COMPOSE_WARNING_LOGS:=false}" && export PODMAN_COMPOSE_WARNING_LOGS + + return 0 +} + +# runtime_is_rootless_podman -> 1 or 0. +# +# Only podman can answer yes: docker's rootless mode still presents a daemon that owns the uid mapping, +# so the ownership problem below does not arise there. Cached because runtime_builder_ids is called on +# the hot path of every build and this shells out. +runtime_is_rootless_podman() { + if [ -z "${_RUNTIME_ROOTLESS:-}" ]; then + if [ "${RUNTIME:-docker}" = podman ] \ + && [ "$(podman info --format '{{.Host.Security.Rootless}}' 2>/dev/null)" = true ]; then + _RUNTIME_ROOTLESS=1 + else + _RUNTIME_ROOTLESS=0 + fi + fi + printf '%s' "$_RUNTIME_ROOTLESS" +} + +# runtime_builder_ids -> the " " pair to run the builder container as, so that the bundle it +# writes into the bind-mounted checkout ends up owned by the person who started the build. +# +# THE ANSWER IS NOT `id -u` EVERYWHERE, and this is the one place the runtimes genuinely diverge. +# +# Docker: the daemon runs as real root, so container uid N writes files owned by host uid N. Passing +# `id -u` is both correct and necessary — without it the bundle comes out root-owned. +# +# Rootless podman: the container is inside a user namespace where the host user is already mapped to +# uid 0, and the subuid range (/etc/subuid, typically 100000+) supplies every other id. So: +# - container uid 0 -> host uid 1000 (the caller — what we want) +# - container uid 1000 -> host uid 101000 (a subuid the caller cannot even write as) +# Passing `id -u` there is actively harmful: measured on podman 5.7, the build cannot create files in +# the mounted checkout at all, and anything it does create is owned by an id the caller cannot chown or +# delete without `podman unshare`. Asking for uid 0 is what yields caller-owned output. +# +# The alternative — `--userns=keep-id` with the real uid — needs a flag that has no Compose equivalent +# here, whereas this needs no change to docker-compose.yml at all. +runtime_builder_ids() { + if [ "$(runtime_is_rootless_podman)" = 1 ]; then + printf '0 0' + else + printf '%s %s' "$(id -u)" "$(id -g)" + fi +} + +# runtime_compose_project -> the compose project name these scripts' containers are labelled with. +# +# Compose's own default rule: the base directory name, lowercased, with everything outside [a-z0-9_-] +# dropped. COMPOSE_PROJECT_NAME overrides it, for both implementations. +runtime_compose_project() { + if [ -n "${COMPOSE_PROJECT_NAME:-}" ]; then printf '%s' "$COMPOSE_PROJECT_NAME"; return; fi + basename "$PWD" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9_-]//g' +} + +# runtime_compose_running_services -> this project's RUNNING compose services, one name per line. +# +# Empty output means "none running". A NON-ZERO RETURN means the question could not be answered at all, +# which callers must not conflate with "none" — see builder_studio_state, whose three-way answer exists +# for exactly this reason. +# +# Two implementations, because `compose ps` is where the CLIs diverge most. docker compose takes +# `--status running --services`; podman-compose 1.5 supports neither (its entire `ps` grammar is `-q` +# and `--format`) and exits 2 with a usage dump. Left as one call, that silently made switch-llm.sh +# conclude the agent was never running and skip the recreate — a wrong answer, not a visible failure. +# +# The fallback asks the RUNTIME rather than the compose wrapper: both implementations stamp the same +# com.docker.compose.{project,service} labels on every container they create (verified against +# podman-compose 1.5), and `ps --filter label=` with `{{.Label "…"}}` behaves identically in both CLIs. +# Trying the native form first means docker's behaviour is unchanged, down to the exit status. +runtime_compose_running_services() { + local out + if out="$("${RUNTIME:?}" compose ps --status running --services 2>/dev/null)"; then + printf '%s\n' "$out" + return 0 + fi + "$RUNTIME" ps --filter "label=com.docker.compose.project=$(runtime_compose_project)" \ + --format '{{.Label "com.docker.compose.service"}}' 2>/dev/null +} + +# runtime_label -> a human name for messages, e.g. "podman (rootless)". Purely cosmetic: a message that +# says "Docker" on a podman box sends people to debug the wrong thing. +runtime_label() { + if [ "$(runtime_is_rootless_podman)" = 1 ]; then printf 'podman (rootless)' + else printf '%s' "${RUNTIME:-docker}"; fi +} + +# ── podman machine sizing ───────────────────────────────────────────────────── +# On macOS and Windows podman runs containers inside a VM whose memory is fixed at machine-create time +# and is NOT elastic the way Docker Desktop's is. That one ceiling is shared by the whole stack (six +# containers) AND by every extension build — and the Angular Native-Federation build is the hungriest +# thing the kit runs: esbuild bundling the federation artifacts is what hits the wall first. +# +# It fails badly enough to be worth pre-empting. The OOM kill surfaces as a bare `Killed`, then ~100 +# lines of Go "fatal error: all goroutines are asleep - deadlock!" out of esbuild, and finally +# `exit status 137` — which is the only part that means "out of memory", and it is the last line anyone +# reads. Spending a few milliseconds here to say it in one sentence is a good trade. +# +# The floor is a JUDGEMENT CALL, not a measurement. The two known data points are: a 2 GiB machine with +# the stack up reliably OOMs the frontend build, and 8 GiB completes it comfortably. Nothing between was +# measured. 6 GiB is set as the hard floor (~4 GiB left for a build once the stack is running) and 8 GiB +# is what the message recommends. Both are overridable in .env for anyone whose workload disagrees. +_RUNTIME_MIN_MEMORY_MIB=6144 +_RUNTIME_MIN_CPUS=4 + +# runtime_machine_check -> 0 when there is nothing to complain about, 1 when the VM is too small to +# build in. Undersized memory is the failure; CPU count only earns a warning, since a slow build still +# produces a bundle. +# +# Scoped to podman deliberately. Docker's VM is managed by Docker Desktop, whose memory is a GUI setting +# this script cannot read. A podman on native Linux has no +# machine at all (`machine list` yields []) and returns 0 — there is no VM to size. +# +# Reads the CONFIGURED size via `machine list`, not the live one via `podman info`: machine list answers +# for a STOPPED machine too (so the check still fires on the very run that would start it), and the +# configured number is the one `podman machine set` changes. +# +# python3, not jq: run.sh hard-requires python3 and never requires jq, and this file is sourced by +# run.sh, stop.sh and logs.sh. Any failure to read or parse returns 0 — this check exists to give a +# better message than exit 137, never to become a new way for the kit to refuse to start. +runtime_machine_check() { + [ "${RUNTIME:-}" = podman ] || return 0 + command -v podman >/dev/null 2>&1 || return 0 + command -v python3 >/dev/null 2>&1 || return 0 + + local min_mem min_cpus raw parsed name mib cpus running + min_mem="$(_runtime_clean "$(_runtime_envv PODMAN_MIN_MEMORY_MIB)")" + min_cpus="$(_runtime_clean "$(_runtime_envv PODMAN_MIN_CPUS)")" + # Fall back to the built-in floor on anything that isn't a plain positive integer, rather than letting + # a typo'd .env value silently disable the check (empty compares as smaller than everything). + case "$min_mem" in ''|*[!0-9]*) min_mem=$_RUNTIME_MIN_MEMORY_MIB ;; esac + case "$min_cpus" in ''|*[!0-9]*) min_cpus=$_RUNTIME_MIN_CPUS ;; esac + + raw="$(podman machine list --format json 2>/dev/null)" || return 0 + [ -n "$raw" ] || return 0 + + parsed="$(printf '%s' "$raw" | python3 -c ' +import json, sys +try: + ms = json.load(sys.stdin) +except Exception: + sys.exit(0) +if not isinstance(ms, list) or not ms: + sys.exit(0) + +# The machine this run will actually talk to: the running one, else the default, else whatever is first. +m = next((x for x in ms if x.get("Running")), None) \ + or next((x for x in ms if x.get("Default")), None) \ + or ms[0] + +try: + mem = int(str(m.get("Memory", "0")).strip() or 0) +except ValueError: + sys.exit(0) +if mem <= 0: + sys.exit(0) +# Current podman reports Memory as a byte count rendered as a STRING; some older builds reported MiB. +# A value below 1 MiB can only be the latter, so treat it as already-MiB rather than rounding to zero. +mib = mem // (1024 * 1024) if mem >= (1 << 20) else mem + +try: + cpus = int(m.get("CPUs") or 0) +except (TypeError, ValueError): + cpus = 0 + +print("%s\t%d\t%d\t%d" % (m.get("Name", "podman-machine-default"), mib, cpus, + 1 if m.get("Running") else 0)) +' 2>/dev/null)" || return 0 + [ -n "$parsed" ] || return 0 + + IFS=$'\t' read -r name mib cpus running <&2 + fi + + [ "$mib" -lt "$min_mem" ] || return 0 + + # What to actually suggest. 8 GiB is the comfortable figure, but a raised PODMAN_MIN_MEMORY_MIB must + # win — otherwise the message demands 16 GiB and then hands over a command that sets 8. + local rec_mem=8192 + [ "$min_mem" -gt "$rec_mem" ] && rec_mem="$min_mem" + + # `podman machine set` writes every flag it is given, so the suggested --cpus must never be LOWER than + # what the machine already has: a 6-CPU machine that is merely short on memory would otherwise be told + # to downgrade itself to the 4-CPU floor. Unknown CPU count (0) falls back to the floor. + local rec_cpus="$min_cpus" + [ "$cpus" -gt "$rec_cpus" ] && rec_cpus="$cpus" + + # The stop/set/start dance is spelled out because `podman machine set --memory` refuses to run against + # a started machine, and because stopping it takes the whole stack down with it — someone mid-session + # deserves to know that before they paste the command, not after. + cat >&2 < 0 Rosetta is live in the podman machine, 1 definitively not, 2 could not tell. +# +# The binfmt handler list is the ONLY honest source. `podman machine inspect` reports the flag the VM was +# last STARTED with, which goes stale the moment containers.conf changes; the handler either exists in the +# running kernel or it does not. The cost is that a stopped or unreachable machine cannot be judged at all +# — hence 2, which the caller treats as "say nothing". +runtime_rosetta_active() { + command -v podman >/dev/null 2>&1 || return 2 + local handlers + handlers="$(podman machine ssh 'ls /proc/sys/fs/binfmt_misc/' 2>/dev/null)" || return 2 + [ -n "$handlers" ] || return 2 + printf '%s\n' "$handlers" | grep -qx 'rosetta' && return 0 + return 1 +} + +# _runtime_machine_vmtype -> the VMType of the machine this run would talk to, or empty. +# +# The impure half of the provider question, split out for the same reason as runtime_rosetta_active: it +# needs podman and a machine, and the policy that consumes it must be testable without either. Same +# machine-selection rule as runtime_machine_check (running, else default, else first) so both checks +# always talk about the same VM. Anything unreadable is empty, never a guess. +_runtime_machine_vmtype() { + command -v podman >/dev/null 2>&1 || { printf ''; return; } + command -v python3 >/dev/null 2>&1 || { printf ''; return; } + podman machine list --format json 2>/dev/null | python3 -c ' +import json, sys +try: + ms = json.load(sys.stdin) +except Exception: + sys.exit(0) +if not isinstance(ms, list) or not ms: + sys.exit(0) +m = next((x for x in ms if x.get("Running")), None) \ + or next((x for x in ms if x.get("Default")), None) \ + or ms[0] +sys.stdout.write(str(m.get("VMType") or "")) +' 2>/dev/null || printf '' +} + +# _runtime_machine_conf_provider -> the [machine] provider in containers.conf, or empty. +_runtime_machine_conf_provider() { + local f="${CONTAINERS_CONF:-$HOME/.config/containers/containers.conf}" + [ -f "$f" ] || { printf ''; return; } + # Same [machine]-table scan as _runtime_rosetta_conf_state: header to the next [section] or EOF. + local block + block="$(awk '/^[[:space:]]*\[machine\][[:space:]]*(#.*)?$/{f=1;next} /^[[:space:]]*\[/{f=0} f' "$f" 2>/dev/null)" + [ -n "$block" ] || { printf ''; return; } + # _runtime_clean strips quotes BEFORE trimming whitespace, so ` "applehv"` (the leading space `cut` + # leaves behind) would keep its quotes. Trim the left side first and the quote is what it sees. + local v + v="$(printf '%s' "$block" | grep -E '^[[:space:]]*provider[[:space:]]*=' | tail -1 | cut -d= -f2-)" + v="${v#"${v%%[![:space:]]*}"}" + _runtime_clean "$v" +} + +# _runtime_machine_provider -> applehv | libkrun | | empty +# +# Which machine provider is in effect, resolved the way podman resolves it: the environment override +# first, then containers.conf, then — for a machine that already exists — what it was actually created +# as. The last one matters because the provider is fixed at `podman machine init`: editing the config +# afterwards changes what the NEXT machine would be, not this one, so the config alone would cheerfully +# report applehv for a libkrun VM that is running right now. +# +# This exists because Rosetta is an applehv feature. Upstream's LibKrunStubber.GetRosetta returns false +# unconditionally, so `rosetta = true` under libkrun is read, discarded, and leaves no trace anywhere +# except the missing binfmt handler — which is exactly the symptom runtime_rosetta_check already sees +# and, until now, mis-diagnosed as "you forgot to restart the machine". +_runtime_machine_provider() { + local p + # An existing machine answers first, and outranks both config sources: `podman machine init + # --provider applehv` does NOT write containers.conf, so the two disagree the moment anyone passes + # that flag — and it is the running VM, not the file, that this run will talk to. Reading the file + # first would blame libkrun for a healthy applehv machine and advise destroying it, which is the + # same class of wrong answer this function exists to stop. + p="$(_runtime_machine_vmtype)" + # No machine yet: both remaining sources describe what the NEXT one would be, in podman's own order. + [ -n "$p" ] || p="$(_runtime_clean "${CONTAINERS_MACHINE_PROVIDER:-}")" + [ -n "$p" ] || p="$(_runtime_machine_conf_provider)" + printf '%s' "$p" +} + +# _runtime_rosetta_conf_state -> absent | no-machine | no-key | disabled | enabled +# +# What containers.conf currently says about Rosetta, so the remediation can be specific. It has to be, +# because the obvious advice is actively harmful: appending a second `[machine]` table to a file that +# already has one is a TOML duplicate-key error, and podman then refuses to do ANYTHING — +# +# Failed to obtain podman configuration: parsing containers.conf: toml: line 11: +# Key 'machine' has already been defined. +# +# — which is a worse place to be than the missing Rosetta we were trying to fix. The `enabled` state +# matters just as much: config that is already right means the machine simply has not been restarted +# since, and telling someone to re-add a key they already have sends them looking in the wrong place. +# +# CONTAINERS_CONF is podman's own override for this path, so honouring it keeps us reading the same +# file podman will, and lets the tests point at a fixture. +_runtime_rosetta_conf_state() { + local f="${CONTAINERS_CONF:-$HOME/.config/containers/containers.conf}" + [ -f "$f" ] || { printf 'absent'; return; } + # The [machine] table runs from its header to the next [section] header or EOF. + local block + block="$(awk '/^[[:space:]]*\[machine\][[:space:]]*(#.*)?$/{f=1;next} /^[[:space:]]*\[/{f=0} f' "$f" 2>/dev/null)" + [ -n "$block" ] || { grep -qE '^[[:space:]]*\[machine\][[:space:]]*(#.*)?$' "$f" 2>/dev/null \ + && { printf 'no-key'; return; }; printf 'no-machine'; return; } + case "$(printf '%s' "$block" | grep -E '^[[:space:]]*rosetta[[:space:]]*=' | tail -1)" in + *true*) printf 'enabled' ;; + *false*) printf 'disabled' ;; + *) printf 'no-key' ;; + esac +} + +# runtime_rosetta_check -> 0 when there is nothing to complain about, 1 when an amd64 studio image is +# about to be handed to QEMU on an Apple Silicon host. +# +# Why this is a hard failure and not a note: QEMU user-mode cannot run the studio's .NET runtime. It +# aborts inside MapControllers() with SIGABRT, the container stays "Up" because the crash does not take +# PID 1 down promptly, /healthz never answers, and run.sh then spends 4.5 minutes on a dot loop before +# failing with "Login failed" — a message about credentials, for a problem that has nothing to do with +# them. Every layer of that is silent or misleading, so it is caught here instead. +# +# Scoped tightly, because Rosetta is only ever relevant to one combination: +# - docker is not checked: Docker Desktop provides Rosetta itself, with nothing to configure. +# - a non-amd64 studio platform needs no translation at all. +# - a non-arm64 host needs no translation to run amd64. +# Anything it cannot determine returns 0. Like runtime_machine_check, this exists to replace a baffling +# failure with a clear one, never to become a new way for the kit to refuse to start. +runtime_rosetta_check() { + [ "${RUNTIME:-}" = podman ] || return 0 + + local plat host rc + # Same precedence as runtime_requested: environment wins over .env. The fallback matches the compose + # default (`platform: ${STUDIO_PLATFORM:-linux/amd64}`), so an unset value is amd64 here too. + plat="$(_runtime_clean "${STUDIO_PLATFORM:-$(_runtime_envv STUDIO_PLATFORM)}")" + [ -n "$plat" ] || plat=linux/amd64 + case "$plat" in *amd64*|*x86_64*) ;; *) return 0 ;; esac + + host="$(_runtime_host_arch)" + [ "$host" = arm64 ] || return 0 + + runtime_rosetta_active; rc=$? + [ "$rc" = 1 ] || return 0 + + # The remediation depends on what containers.conf already says. This is not polish: telling someone to + # append `[machine]` to a file that already has that table produces a TOML duplicate-key error, and + # podman then refuses to run at all ("Key 'machine' has already been defined") — strictly worse than + # the missing Rosetta. And when the key is already correct, the file is not the problem: the machine + # simply has not been restarted since it was set. + local conf_path conf_dir fix provider after + conf_path="${CONTAINERS_CONF:-$HOME/.config/containers/containers.conf}" + conf_dir="$(dirname "$conf_path")" + provider="$(_runtime_machine_provider)" + + # The provider outranks every conf state, because under anything but applehv the rosetta key is read + # and thrown away: "your config is already correct, just restart" is then advice that cannot ever come + # true, and the person dutifully recreates the machine again and again with the same result. + if [ -n "$provider" ] && [ "$provider" != applehv ]; then + fix="The machine provider is '$provider', and Rosetta is an applehv feature. podman's $provider +backend reports Rosetta as unavailable whatever containers.conf says, so a 'rosetta = true' +there is read and discarded — which is why this looks like a config that ought to work. + +Set BOTH keys under the SINGLE existing [machine] section of +$conf_path +— never add a second [machine] header, TOML rejects a duplicate table and podman then +refuses to run at all: + + [machine] + provider = \"applehv\" + rosetta = true" + after="Then recreate the machine. The PROVIDER, unlike the rosetta key, is written at +'podman machine init' and never revisited, so it cannot be changed in place. +Recreating DESTROYS that VM's images and volumes: + + podman machine stop + podman machine rm -f + podman machine init --provider applehv --cpus 4 -m 8192 --disk-size 100 + podman machine start + ./run.sh" + else + case "$(_runtime_rosetta_conf_state)" in + absent) + fix="No $conf_path yet — create it: + + mkdir -p $conf_dir + printf '[machine]\\nrosetta = true\\n' > $conf_path" ;; + no-machine) + fix="$conf_path exists but has no [machine] section. Open it and ADD this section +(do not append a second one elsewhere in the file): + + [machine] + rosetta = true" ;; + no-key) + fix="$conf_path already has a [machine] section. Add this line UNDER that existing +section — do not add a second [machine] header, TOML rejects a duplicate table and podman +then refuses to start at all: + + rosetta = true" ;; + disabled) + fix="$conf_path sets 'rosetta = false' under [machine]. Change that one value to: + + rosetta = true" ;; + enabled) + fix="$conf_path ALREADY sets 'rosetta = true' — the file is correct and needs no edit. +The key is read at every machine start, so the machine simply has not been restarted since." ;; + esac + after="Then restart the machine — it does NOT need to be recreated, and nothing is lost. applehv +re-reads the rosetta key from containers.conf on every start and syncs it into the machine: + + podman machine stop + podman machine start + ./run.sh" + fi + + cat >&2 < # Only meaningful if the role itself checked out. if [ "$RC" = 0 ]; then - if ! command -v docker >/dev/null 2>&1; then - echo "CONTAINER_IMDS=unknown:docker not installed" - elif ! docker info >/dev/null 2>&1; then - echo "CONTAINER_IMDS=unknown:docker daemon not reachable" + # Quietly: this probe is one line of a diagnostic report, and a runtime that cannot be resolved is + # already reported as unknown: below rather than as a failure of the whole report. + if ! runtime_resolve 2>/dev/null; then + echo "CONTAINER_IMDS=unknown:no container runtime available" + elif ! "$RUNTIME" info >/dev/null 2>&1; then + echo "CONTAINER_IMDS=unknown:$RUNTIME not reachable" else - # The probe runs BEFORE `docker compose pull`, so on a fresh box this image is usually not local + # The probe runs BEFORE `$RUNTIME compose pull`, so on a fresh box this image is usually not local # yet. Pull it quietly rather than reporting a false "blocked". - docker image inspect busybox:1.36 >/dev/null 2>&1 || docker pull -q busybox:1.36 >/dev/null 2>&1 || true - if ! docker image inspect busybox:1.36 >/dev/null 2>&1; then + "$RUNTIME" image inspect busybox:1.36 >/dev/null 2>&1 || "$RUNTIME" pull -q busybox:1.36 >/dev/null 2>&1 || true + if ! "$RUNTIME" image inspect busybox:1.36 >/dev/null 2>&1; then echo "CONTAINER_IMDS=unknown:could not obtain busybox:1.36 (registry unreachable?)" - elif docker run --rm busybox:1.36 sh -c \ + elif "$RUNTIME" run --rm busybox:1.36 sh -c \ 'wget -S -T 3 -O /dev/null http://169.254.169.254/latest/meta-data/ 2>&1 | grep -q "HTTP/"' \ >/dev/null 2>&1; then echo "CONTAINER_IMDS=ok" diff --git a/scripts/switch-llm.sh b/scripts/switch-llm.sh index 9124c57..eb0ab50 100755 --- a/scripts/switch-llm.sh +++ b/scripts/switch-llm.sh @@ -30,8 +30,10 @@ set -euo pipefail cd "$(dirname "$0")/.." ENV=.env [ -f "$ENV" ] || { echo "No .env found — run ./run.sh first to set up the stack." >&2; exit 1; } +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman) . ./scripts/_provider_gateway.sh . ./scripts/_provider_subscription.sh +runtime_resolve || exit 1 NONINTERACTIVE=0 F_ANTHROPIC=""; F_AWS_KEY=""; F_AWS_SECRET=""; F_AWS_TOKEN=""; F_AWS_REGION="" @@ -221,13 +223,13 @@ esac setenv DEVKIT_MODEL "$TARGET" -if ! docker compose ps --status running --services 2>/dev/null | grep -qx claude-code-agent; then +if ! runtime_compose_running_services | grep -qx claude-code-agent; then echo "==> Agent container isn't running — .env is updated; start the stack with ./run.sh to apply it." >&2 exit 0 fi echo "==> Recreating the agent with the new provider…" -docker compose up -d claude-code-agent +"$RUNTIME" compose up -d claude-code-agent echo "==> Registering $(getenv CLAUDE_MODEL) ($LLM_DESC) as the System default LLM…" if LLM_PROVIDER_LABEL="$LLM_DESC" ./scripts/register-llm.sh; then diff --git a/stop.sh b/stop.sh index 47d7ff6..903b6f1 100755 --- a/stop.sh +++ b/stop.sh @@ -2,11 +2,13 @@ # Stop the platform. Volumes are kept (loaded extensions + data survive). Pass --wipe to remove them. set -euo pipefail cd "$(dirname "$0")" +. ./scripts/_runtime.sh # → $RUNTIME (docker | podman) +runtime_resolve || exit 1 if [ "${1:-}" = "--wipe" ]; then echo "==> Stopping and REMOVING volumes (extensions, mongo, file store will be lost)…" - docker compose down -v + "$RUNTIME" compose down -v else echo "==> Stopping (volumes kept; extensions replay on next ./run.sh)…" - docker compose down + "$RUNTIME" compose down fi diff --git a/tests/test-runtime.sh b/tests/test-runtime.sh new file mode 100755 index 0000000..d32a89e --- /dev/null +++ b/tests/test-runtime.sh @@ -0,0 +1,511 @@ +#!/usr/bin/env bash +# Checks the container-runtime abstraction in scripts/_runtime.sh, and that nothing in the kit has +# gone back to calling `docker` directly. +# +# Static + pure-function checks only: nothing here starts the stack, pulls an image, or needs any +# particular runtime to be installed — so it runs the same on a docker laptop, a podman box and CI. +set -uo pipefail +cd "$(dirname "$0")/.." || exit 1 +PASS=0; FAIL=0 +t() { printf ' %s … ' "$1"; } +ok() { echo "ok"; PASS=$((PASS+1)); } +bad() { echo "FAIL: $1"; FAIL=$((FAIL+1)); } + +# Resolution is exercised in a subshell per case: runtime_resolve exports RUNTIME and memoises itself, +# so cases would otherwise contaminate each other. DUPLO_ENV_FILE=/dev/null keeps a developer's real +# .env (which may pin RUNTIME) out of the auto-detect cases. +res() { # res -> prints "|" + ( eval "$1"; export DUPLO_ENV_FILE=/dev/null + . ./scripts/_runtime.sh + if runtime_resolve 2>/dev/null; then printf '0|%s' "$RUNTIME"; else printf '1|'; fi ) +} + +echo "container runtime abstraction:" + +t "_runtime.sh is syntactically valid and sourcing it defines the API" +if bash -n scripts/_runtime.sh 2>/dev/null && \ + ( . ./scripts/_runtime.sh + for f in runtime_detect runtime_resolve runtime_requested runtime_builder_ids \ + runtime_is_rootless_podman runtime_compose_project runtime_compose_running_services \ + runtime_label runtime_rosetta_check; do + declare -F "$f" >/dev/null || exit 1 + done ); then ok; else bad "missing function(s)"; fi + +t "every supported runtime is accepted when present on PATH" +MISS="" +for r in docker podman; do + command -v "$r" >/dev/null 2>&1 || continue # only assert about what's installed here + [ "$(res "RUNTIME=$r")" = "0|$r" ] || MISS="$MISS $r" +done +if [ -z "$MISS" ]; then ok; else bad "rejected:$MISS"; fi + +t "runc and friends are rejected as low-level OCI runtimes, not silently accepted" +MISS="" +for r in runc crun youki runsc; do + [ "$(res "RUNTIME=$r")" = "1|" ] || MISS="$MISS $r" +done +if [ -z "$MISS" ]; then ok; else bad "not rejected:$MISS"; fi + +t "the runc rejection explains itself rather than saying 'unsupported'" +MSG="$( ( export DUPLO_ENV_FILE=/dev/null RUNTIME=runc + . ./scripts/_runtime.sh; runtime_resolve ) 2>&1 )" +if grep -q 'low-level OCI runtime' <<<"$MSG" && grep -q 'compose' <<<"$MSG" && grep -q 'podman' <<<"$MSG" +then ok; else bad "unhelpful message: $MSG"; fi + +t "an unknown runtime is rejected" +if [ "$(res "RUNTIME=nosuchruntime")" = "1|" ]; then ok; else bad "accepted"; fi + +t "a requested-but-absent runtime is rejected rather than falling back to another" +# Must name a SUPPORTED runtime that happens not to be installed here — an unsupported name would be +# rejected by the wrong branch and the test would pass while proving nothing. Which one that is differs +# per machine, so find it rather than hard-coding one. +ABSENT="" +for r in docker podman; do + command -v "$r" >/dev/null 2>&1 || { ABSENT="$r"; break; } +done +if [ -z "$ABSENT" ]; then ok # every supported runtime is installed here +elif [ "$(res "RUNTIME=$ABSENT")" = "1|" ]; then ok +else bad "fell back instead of failing"; fi + +t "values are cleaned of quotes, whitespace and CRs before use" +R="$(res 'RUNTIME=" podman "')" +if ! command -v podman >/dev/null 2>&1; then ok # nothing to assert on a podman-less box +elif [ "$R" = "0|podman" ]; then ok; else bad "got '$R'"; fi + +t "auto-detect picks an installed runtime, preferring docker" +R="$(res 'unset RUNTIME')" +EXPECT=""; for r in docker podman; do command -v "$r" >/dev/null 2>&1 && { EXPECT="$r"; break; }; done +if [ -z "$EXPECT" ]; then ok # no runtime installed: nothing to prefer +elif [ "$R" = "0|$EXPECT" ]; then ok; else bad "got '$R', expected '0|$EXPECT'"; fi + +t "runtime_requested distinguishes an explicit choice from auto-detect" +A="$( ( export DUPLO_ENV_FILE=/dev/null; unset RUNTIME; . ./scripts/_runtime.sh; runtime_requested ) )" +B="$( ( export DUPLO_ENV_FILE=/dev/null RUNTIME=podman; . ./scripts/_runtime.sh; runtime_requested ) )" +if [ -z "$A" ] && [ "$B" = podman ]; then ok; else bad "auto='$A' explicit='$B'"; fi + +# Rosetta preflight. The two impure parts — what the host CPU is, and whether the podman machine has a +# rosetta binfmt handler — are separate functions precisely so the POLICY can be exercised here without +# podman, a VM, or an Apple Silicon host. Redefining them after sourcing is the seam. +ros() { # ros [runtime] -> "|" + ( HA="$1"; PL="$2"; PRC="$3" + export DUPLO_ENV_FILE=/dev/null RUNTIME="${4:-podman}" STUDIO_PLATFORM="$PL" + # Pin the provider inputs too: left alone, the check reads the developer's real containers.conf and + # shells out to podman, and these cases would then answer differently on an applehv laptop than on + # CI. The provider's own behaviour is asserted through rosmsg below, where it is set explicitly. + export CONTAINERS_CONF=/dev/null; unset CONTAINERS_MACHINE_PROVIDER + . ./scripts/_runtime.sh + eval "_runtime_host_arch() { printf '%s' '$HA'; }" + eval "runtime_rosetta_active() { return $PRC; }" + _runtime_machine_vmtype() { printf ''; } + ERR="$(runtime_rosetta_check 2>&1 >/dev/null)"; RC=$? + printf '%s|%s' "$RC" "$(printf '%s' "$ERR" | head -1)" ) +} + +# The remediation the check prints depends on what containers.conf already says. Appending a second +# [machine] table to an existing file is a TOML "key already defined" error that stops podman dead, so +# the advice must never be one-size-fits-all. CONTAINERS_CONF is podman's own override, so pointing it +# at a fixture is both faithful and testable. +conf() { # conf -> the state word + ( F=/dev/null + if [ "$1" != NONE ]; then F="$(mktemp)"; printf '%s' "$1" > "$F"; else F="$(mktemp -u)"; fi + export CONTAINERS_CONF="$F" + . ./scripts/_runtime.sh + _runtime_rosetta_conf_state ) +} + +t "conf state reports absent when containers.conf does not exist" +[ "$(conf NONE)" = absent ] && ok || bad "got '$(conf NONE)'" + +t "conf state reports enabled when [machine] already sets rosetta = true" +R="$(conf '[containers] +[machine] +rosetta = true +[network] +')" +[ "$R" = enabled ] && ok || bad "got '$R'" + +t "conf state reports disabled when [machine] explicitly sets rosetta = false" +R="$(conf '[machine] +rosetta = false +')" +[ "$R" = disabled ] && ok || bad "got '$R'" + +t "conf state reports no-key when [machine] exists without a rosetta key" +R="$(conf '[containers] +[machine] +cpus = 4 +')" +[ "$R" = no-key ] && ok || bad "got '$R'" + +t "conf state finds [machine] when the header carries a trailing comment" +# `[machine] # my settings` is valid TOML. Anchoring the scan on end-of-line after the header hid the +# section, and the resulting no-machine advice tells the user to ADD [machine] — a duplicate-key error +# that stops podman outright, on a user whose config was already correct. +R="$(conf '[containers] +[machine] # my settings +rosetta = true +')" +[ "$R" = enabled ] && ok || bad "got '$R' (a commented header must not read as a missing section)" + +t "conf state reports no-machine when the file exists with no [machine] section" +R="$(conf '[containers] +[engine] +env = [] +')" +[ "$R" = no-machine ] && ok || bad "got '$R'" + +# Machine provider. Rosetta is an applehv feature: the libkrun provider's GetRosetta returns false +# unconditionally upstream, so `rosetta = true` under libkrun is read and discarded. The provider is +# therefore part of the diagnosis, not a detail. Resolution order mirrors podman's own, and the podman +# call is its own function so the policy is testable without a VM. +prov() { # prov -> the provider word + ( eval "$1" + if [ "$2" != NONE ]; then F="$(mktemp)"; printf '%s' "$2" > "$F"; else F="$(mktemp -u)"; fi + export CONTAINERS_CONF="$F" + . ./scripts/_runtime.sh + if [ "$3" != NONE ]; then eval "_runtime_machine_vmtype() { printf '%s' '$3'; }" + else _runtime_machine_vmtype() { printf ''; }; fi + _runtime_machine_provider ) +} + +t "an existing machine's VMType outranks containers.conf, which only describes the NEXT machine" +# `podman machine init --provider applehv` does not write containers.conf, so the two disagree the +# moment anyone passes the flag — and the VM that is running is the one this run will talk to. Reading +# the config first would blame libkrun for a healthy applehv machine and advise destroying it. +R="$(prov 'unset CONTAINERS_MACHINE_PROVIDER' '[machine] +provider = "libkrun" +' applehv)" +[ "$R" = applehv ] && ok || bad "got '$R'" + +t "with no machine yet, CONTAINERS_MACHINE_PROVIDER outranks containers.conf" +R="$(prov 'export CONTAINERS_MACHINE_PROVIDER=applehv' '[machine] +provider = "libkrun" +' NONE)" +[ "$R" = applehv ] && ok || bad "got '$R'" + +t "with no machine and no environment override, containers.conf says what the next machine will be" +R="$(prov 'unset CONTAINERS_MACHINE_PROVIDER' '[machine] +provider = "libkrun" +' NONE)" +[ "$R" = libkrun ] && ok || bad "got '$R'" + +t "machine provider is empty rather than guessed when nothing can say" +R="$(prov 'unset CONTAINERS_MACHINE_PROVIDER' NONE NONE)" +[ -z "$R" ] && ok || bad "got '$R'" + +t "rosetta check fails when an amd64 image would run on an arm64 host without Rosetta" +R="$(ros arm64 linux/amd64 1)" +case "$R" in 1\|*[Rr]osetta*) ok ;; *) bad "got '$R'" ;; esac + +rosmsg() { # rosmsg [provider] -> the whole stderr message + ( C="$1"; P="${2-}" + if [ "$C" != NONE ]; then F="$(mktemp)"; printf '%s' "$C" > "$F"; else F="$(mktemp -u)"; fi + export DUPLO_ENV_FILE=/dev/null RUNTIME=podman STUDIO_PLATFORM=linux/amd64 CONTAINERS_CONF="$F" + . ./scripts/_runtime.sh + _runtime_host_arch() { printf 'arm64'; } + runtime_rosetta_active() { return 1; } + # Unset provider (the default) is the "cannot tell" case: the advice must still stand on the config + # alone, which is what every pre-existing case below asserts. + eval "_runtime_machine_provider() { printf '%s' '$P'; }" + runtime_rosetta_check 2>&1 >/dev/null ) +} + +# NOTE: capture into a variable, never `rosmsg ... | grep -q`. Under the `set -o pipefail` at the top of +# this file, grep -q exits on first match, SIGPIPEs the writer, and the pipeline reports the writer's +# failure — so the condition reads FALSE precisely when the pattern matched. That silently inverted +# these three assertions when they were first written. +t "the fix never suggests appending a second [machine] table to an existing config" +M="$(rosmsg '[containers] +[machine] +rosetta = false +')" +if printf '%s\n' "$M" | grep -qE '>>[[:space:]]*[^|]*containers\.conf'; then + bad "tells the user to append a second [machine] table — a TOML duplicate-key error that stops podman" +else ok; fi + +t "an applehv machine whose config is already right points at a restart, not at recreating" +# applehv's StartVM re-reads cfg.Machine.Rosetta from containers.conf on every start and syncs it into +# the machine config, so the key really does take effect without re-initialising. Only the PROVIDER is +# fixed at init. Telling an applehv user to destroy their VM would cost them images and volumes for a +# change a stop/start already makes. +M="$(rosmsg '[machine] +rosetta = true +' applehv)" +if printf '%s\n' "$M" | grep -qi 'already' && printf '%s\n' "$M" | grep -q 'podman machine start' \ + && ! printf '%s\n' "$M" | grep -q 'podman machine rm'; then ok +else bad "got: $(printf '%s' "$M" | sed -n '9,16p' | tr '\n' ' ')"; fi + +t "a libkrun machine is diagnosed as the provider, not as a missing or unrestarted config" +M="$(rosmsg '[machine] +provider = "libkrun" +rosetta = true +' libkrun)" +if printf '%s\n' "$M" | grep -q 'applehv' && printf '%s\n' "$M" | grep -qi 'libkrun'; then ok +else bad "never names the provider: $(printf '%s' "$M" | sed -n '5,14p' | tr '\n' ' ')"; fi + +t "the libkrun remediation does not send the user back to a config that is already correct" +M="$(rosmsg '[machine] +provider = "libkrun" +rosetta = true +' libkrun)" +if printf '%s\n' "$M" | grep -qi 'rosetta = true.*add\|add.*rosetta = true'; then + bad "tells them to add a key they already have" +else ok; fi + +t "a non-applehv machine is never told a restart will do, because the provider is fixed at init" +# The provider — unlike rosetta — is written at `podman machine init` and never revisited. The old +# footer's "it does NOT need to be recreated, and nothing is lost" is true for applehv and false here, +# and it is the sentence that turns a five-minute fix into an afternoon of recreating the same VM. +M="$(rosmsg '[machine] +provider = "libkrun" +rosetta = true +' libkrun)" +if printf '%s\n' "$M" | grep -qi 'not need to be recreated'; then + bad "promises a restart is enough for a provider that can never honour the key" +elif printf '%s\n' "$M" | grep -q 'podman machine rm'; then ok +else bad "never tells them to recreate: $(printf '%s' "$M" | sed -n '20,28p' | tr '\n' ' ')"; fi + +t "a missing config gets a create command that cannot clobber anything" +M="$(rosmsg NONE)" +# `>` not `>>`: there is no file to append to, and `>>` would be the wrong habit to teach. +if printf '%s\n' "$M" | grep -qE "printf .* > " && printf '%s\n' "$M" | grep -qi 'create it'; then ok +else bad "no safe create command: $(printf '%s' "$M" | sed -n '10,13p' | tr '\n' ' ')"; fi + +t "rosetta check is a no-op on docker, which provides Rosetta itself" +R="$(ros arm64 linux/amd64 1 docker)" +[ "$R" = "0|" ] && ok || bad "got '$R'" + +t "rosetta check is a no-op for an arm64 studio image, which needs no translation" +R="$(ros arm64 linux/arm64 1)" +[ "$R" = "0|" ] && ok || bad "got '$R'" + +t "rosetta check is a no-op on an amd64 host, which needs no translation" +R="$(ros amd64 linux/amd64 1)" +[ "$R" = "0|" ] && ok || bad "got '$R'" + +t "rosetta check is a no-op when Rosetta is active" +R="$(ros arm64 linux/amd64 0)" +[ "$R" = "0|" ] && ok || bad "got '$R'" + +t "rosetta check stays quiet when the machine cannot be probed rather than crying wolf" +R="$(ros arm64 linux/amd64 2)" +[ "$R" = "0|" ] && ok || bad "got '$R'" + +t "rosetta check treats an unset STUDIO_PLATFORM as amd64, matching the compose default" +R="$(ros arm64 "" 1)" +case "$R" in 1\|*[Rr]osetta*) ok ;; *) bad "got '$R'" ;; esac + +t "builder ids are the caller's uid/gid on docker, but 0:0 on rootless podman" +IDS="$( ( export DUPLO_ENV_FILE=/dev/null; . ./scripts/_runtime.sh; runtime_resolve 2>/dev/null + runtime_builder_ids ) )" +if [ -z "$IDS" ]; then bad "no ids" +elif [ "$( ( export DUPLO_ENV_FILE=/dev/null; . ./scripts/_runtime.sh + runtime_resolve 2>/dev/null; runtime_is_rootless_podman ) )" = 1 ]; then + [ "$IDS" = "0 0" ] && ok || bad "rootless podman should be '0 0', got '$IDS'" +else + [ "$IDS" = "$(id -u) $(id -g)" ] && ok || bad "expected '$(id -u) $(id -g)', got '$IDS'" +fi + +echo +echo +echo "runtime failure messages:" + +# runtime_alternatives asks, of the runtimes we did NOT pick, which ones actually answer. The reachability +# probe is its own function so the policy is testable with no runtime installed at all — the same seam +# runtime_rosetta_active provides for the Rosetta check. +alts() { # alts -> runtime_alternatives output + ( export DUPLO_ENV_FILE=/dev/null RUNTIME="$1"; ANS=" $2 " + . ./scripts/_runtime.sh + eval '_runtime_answers() { case "$ANS" in *" $1 "*) return 0 ;; *) return 1 ;; esac; }' + runtime_alternatives ) +} + +t "alternatives never include the runtime we already resolved to" +[ "$(alts docker 'docker podman')" = podman ] && ok || bad "got '$(alts docker 'docker podman')'" + +t "alternatives list only runtimes that actually answer" +[ "$(alts docker '')" = "" ] && ok || bad "got '$(alts docker '')' for a box where nothing answers" + +t "alternatives are empty when the only other runtime is installed but dead" +[ "$(alts podman 'podman')" = "" ] && ok || bad "got '$(alts podman 'podman')'" + +# A compose-capable alternative is a narrower question than a reachable one: a runtime can answer and +# still have no `compose` subcommand, in which case naming it as the way out would send the user in a +# circle. Both probes are separate functions so this is testable with neither runtime installed. +altsc() { # altsc -> runtime_alternatives_compose output + ( export DUPLO_ENV_FILE=/dev/null RUNTIME="$1"; ANS=" $2 "; CMP=" $3 " + . ./scripts/_runtime.sh + eval '_runtime_answers() { case "$ANS" in *" $1 "*) return 0 ;; *) return 1 ;; esac; }' + eval '_runtime_has_compose() { case "$CMP" in *" $1 "*) return 0 ;; *) return 1 ;; esac; }' + runtime_alternatives_compose ) +} + +t "compose alternatives name a runtime that both answers and has compose" +[ "$(altsc podman 'docker podman' 'docker')" = docker ] && ok || bad "got '$(altsc podman 'docker podman' 'docker')'" + +t "compose alternatives exclude a runtime that answers but has no compose" +[ "$(altsc podman 'docker' '')" = "" ] && ok || bad "got '$(altsc podman 'docker' '')' — naming it sends the user in a circle" + +t "compose alternatives never include the runtime we already resolved to" +[ "$(altsc docker 'docker' 'docker')" = "" ] && ok || bad "got '$(altsc docker 'docker' 'docker')'" + +t "runtime guidance points at .env, never a one-shot on the command line" +# A CLI-scoped RUNTIME lasts exactly one process. run.sh, build-extension.sh, build-all.sh, logs.sh and +# stop.sh each resolve independently, so a one-off gets you past one command and leaves the next on the +# other runtime — which is precisely how the cross-runtime build warning happens. .env is the only place +# every script reads, so it is the only honest advice. +HITS="$(grep -nE "RUNTIME=[^ \"']+ +\./" run.sh scripts/_builder.sh scripts/_runtime.sh 2>/dev/null)" +if [ -z "$HITS" ]; then ok; else bad "recommends a per-process override: $HITS"; fi + +t "runtime guidance says why .env and not per-command" +if grep -qiE 'each script resolves|every script reads|resolve.*independently' run.sh scripts/_builder.sh; then ok +else bad "tells the user where to set it but not why the command line will not do"; fi + +t "run.sh's compose preflight names a compose-capable alternative" +# Hit in practice: .env pinned RUNTIME=podman, podman's socket was broken, docker was up with compose +# v2 — and the failure listed install advice for both runtimes without saying docker would just work. +if grep -B4 -A14 "compose version' failed" run.sh | grep -q 'runtime_alternatives_compose'; then ok +else bad "compose preflight still recites prerequisites instead of naming the working runtime"; fi + +# The message itself is pure: given what was resolved, what else answers, and what the toolchain lacks, +# it only formats. No probing, so every case below is exercised on any machine. +msg() { # msg -> the whole message + ( export DUPLO_ENV_FILE=/dev/null + . ./scripts/_builder.sh + builder_runtime_message "$1" "$2" "$3" "$4" 2>&1 ) +} + +t "case A: an auto-detected but dead runtime names the working alternative and how to use it" +M="$(msg docker podman dotnet 0)" +if grep -q 'podman' <<<"$M" && grep -q 'RUNTIME=podman' <<<"$M" && grep -qiE 'not responding' <<<"$M" +then ok; else bad "$M"; fi + +t "case A: it never tells you to install the runtime you already have" +M="$(msg docker podman dotnet 0)" +if grep -qiE 'install docker|install docker or podman' <<<"$M"; then + bad "tells the user to install docker, which is installed — the whole bug" +else ok; fi + +t "case A-prime: an EXPLICITLY requested runtime is not described as auto-detected" +M="$(msg docker podman dotnet 1)" +if grep -qi 'auto-detected' <<<"$M"; then bad "implies they did not choose it: $M" +elif grep -qiE 'requested|asked for' <<<"$M" && grep -q 'RUNTIME=podman' <<<"$M"; then ok +else bad "$M"; fi + +t "case B: a dead runtime with no alternative says to START it, not to install it" +M="$(msg docker '' dotnet 0)" +if grep -qiE 'install docker' <<<"$M"; then bad "says install, but docker is present: $M" +elif grep -qiE 'start' <<<"$M" && grep -qiE 'not responding|installed but' <<<"$M"; then ok +else bad "$M"; fi + +t "case B: the --native escape hatch is offered and names what is missing" +M="$(msg docker '' dotnet 0)" +if grep -q -- '--native' <<<"$M" && grep -q 'dotnet' <<<"$M"; then ok; else bad "$M"; fi + +t "case C: with nothing on PATH at all, installing IS the right advice" +M="$(msg '' '' dotnet 0)" +if grep -qiE 'install' <<<"$M" && grep -qiE 'on PATH|found' <<<"$M"; then ok; else bad "$M"; fi + +t "case C is distinguishable from case B in the first line" +if [ "$(msg '' '' dotnet 0 | head -1)" != "$(msg docker '' dotnet 0 | head -1)" ]; then ok +else bad "'nothing installed' and 'installed but stopped' open identically"; fi + +t "run.sh's not-ready warning names a working alternative when there is one" +if grep -A6 'doesn.t look ready\|not responding' run.sh | grep -q 'runtime_alternatives'; then ok +else bad "run.sh warns the runtime is dead without mentioning the healthy one sitting next to it"; fi + +t "run.sh says which runtime it picked on the happy path, not only on failure" +# Presence-based auto-detect means the choice is invisible until something breaks; one line fixes that. +if grep -B4 'Pulling images' run.sh | grep -qE 'runtime_label|using .*RUNTIME|\$RUNTIME"?\)'; then ok +else bad "no runtime announced next to 'Pulling images'"; fi + +t "builder_dispatch's error:runtime branch delegates to the message function" +if grep -A4 'error:runtime)' scripts/_builder.sh | grep -q 'builder_runtime_message'; then ok +else bad "the branch still inlines its own text, so none of the above applies to a real build"; fi + +echo "no stray hard-coded docker calls:" + +# The scripts that drive containers. Anything invoking the CLI must go through $RUNTIME so that the +# RUNTIME setting is actually honoured — a single stray `docker compose` silently ignores it. +# +# Detection strips comments AND double-quoted strings before matching: the kit legitimately NAMES these +# CLIs in help and error text ("Install docker, podman, …"), and only an actual invocation matters. A +# command name is never itself inside quotes, so dropping quoted spans cannot hide a real call. The +# `exec`/`command` prefix is matched explicitly — `exec docker compose` is exactly how logs.sh and the +# build dispatcher invoke it, and a check that missed that would pass while testing nothing. +# +# nerdctl and finch stay in the alternation below even though neither is a SUPPORTED runtime: this is a +# lint against hard-coding any CLI, and hard-coding an unsupported one is worse, not better. +t "no script invokes 'docker'/'podman' as a command instead of \$RUNTIME" +STRAY="" +for f in run.sh stop.sh logs.sh scripts/*.sh tests/*.sh; do + [ "$f" = tests/test-runtime.sh ] && continue # this file names them on purpose + sed -e 's/[[:space:]]*#.*$//' -e 's/"[^"]*"//g' "$f" \ + | grep -qE '(^|[^-[:alnum:]_./$])((exec|command)[[:space:]]+)?(docker|podman|nerdctl|finch)[[:space:]]+(compose|run|pull|build|images?|ps|info|version|rmi?|logs|volume|inspect|exec)\b' \ + && STRAY="$STRAY $f" +done +if [ -z "$STRAY" ]; then ok; else bad "hard-coded CLI in:$STRAY"; fi + +t "run.sh runs both podman preflights, and after RUNTIME is resolved" +if MC=$(grep -n '^runtime_machine_check' run.sh | cut -d: -f1) && \ + RC=$(grep -n '^runtime_rosetta_check' run.sh | cut -d: -f1) && \ + RES=$(grep -n 'runtime_resolve' run.sh | head -1 | cut -d: -f1) && \ + [ -n "$MC" ] && [ -n "$RC" ] && [ "$RES" -lt "$MC" ] && [ "$RES" -lt "$RC" ]; then ok +else bad "machine=${MC:-absent} rosetta=${RC:-absent} resolve=${RES:-absent}"; fi + +t "every script that runs containers sources _runtime.sh and resolves before using \$RUNTIME" +MISS="" +for f in run.sh stop.sh logs.sh scripts/_builder.sh scripts/switch-llm.sh scripts/detect-bedrock.sh; do + grep -q '_runtime.sh' "$f" || MISS="$MISS $f" +done +if [ -z "$MISS" ]; then ok; else bad "not sourced in:$MISS"; fi + +t "the builder run names its compose profile (podman-compose won't infer it)" +if grep -q 'compose --profile tools run' scripts/_builder.sh; then ok +else bad "scripts/_builder.sh must pass --profile tools"; fi + +t "docker-compose.yml has no nested \${...\${...}} (podman-compose mis-parses it)" +# Comments are stripped first: the comment above that line documents the broken form on purpose, and +# a comment cannot affect how compose interpolates anything. +if sed 's/[[:space:]]*#.*$//' docker-compose.yml | grep -qE '\$\{[^}]*\$\{'; then + bad "nested interpolation present"; else ok; fi + +t ".env.example documents RUNTIME and says runc is not a valid value" +if grep -q '^#RUNTIME=' .env.example && grep -qi 'runc' .env.example; then ok; else bad ".env.example"; fi + +t "run.sh creates .env before it resolves the runtime" +# runtime_resolve reads RUNTIME from .env and memoises itself, so if .env is created after it runs, a +# first run on a box with both CLIs installed auto-detects docker and ignores RUNTIME=podman entirely -- +# for both preflights, `compose pull` and `compose up`. +ENVL="$(grep -n 'cp .env.example' run.sh | head -1 | cut -d: -f1)" +RESL="$(grep -nE '^runtime_resolve' run.sh | head -1 | cut -d: -f1)" +if [ -n "$ENVL" ] && [ -n "$RESL" ] && [ "$ENVL" -lt "$RESL" ]; then ok +else bad ".env created at line ${ENVL:-?} but runtime resolved at ${RESL:-?} -- RUNTIME in .env is ignored on a first run"; fi + +t "podman compose guidance names more than one way to get a provider" +# podman needs *a* compose provider, not specifically podman-compose: a docker-compose binary satisfies +# `podman compose` just as well, and Podman Desktop installs one. Naming only the brew package sends +# people to install a second provider they may already have under another name. +MISS="" +for f in run.sh docs/getting-started/prerequisites.md; do + if grep -q 'podman-compose' "$f" 2>/dev/null; then + grep -qi 'podman desktop' "$f" 2>/dev/null || MISS="$MISS $f" + fi +done +if [ -z "$MISS" ]; then ok; else bad "names only podman-compose, no alternative route:$MISS"; fi + +t "runtime install guidance names brew only, not apt or dnf" +# Linux is not supported right now, so apt/dnf guidance sends the large majority of users to a command +# that does not exist on their machine. +# Scoped to the podman-compose guidance this PR added; the pre-existing python3 hint is not ours. +HITS="$(grep -n 'podman-compose' run.sh docs/getting-started/prerequisites.md 2>/dev/null | grep -E 'apt|dnf')" +if [ -z "$HITS" ]; then ok; else bad "non-brew install guidance: $HITS"; fi + +t "bash -n on every shell script in the kit" +BADF="" +for f in run.sh stop.sh logs.sh scripts/*.sh tests/*.sh; do bash -n "$f" 2>/dev/null || BADF="$BADF $f"; done +if [ -z "$BADF" ]; then ok; else bad "syntax:$BADF"; fi + +echo +echo "$PASS passed, $FAIL failed" +[ "$FAIL" -eq 0 ]