feat(agent-challenge): CVM sizing, token handoff, and Phala eval deploy hardening

packages/challenges/agent-challenge/docker/review/phala_pre_launch.sh

@@ -59,7 +59,7 @@ echo "Starting login process..."
 # Check if Docker credentials exist
 if [[ -n "$DSTACK_DOCKER_USERNAME" && -n "$DSTACK_DOCKER_PASSWORD" ]]; then
     echo "Docker credentials found"
-    DOCKER_REGISTRY_TARGET="${DSTACK_DOCKER_REGISTRY:-docker.io}"
+    DOCKER_REGISTRY_TARGET="${DSTACK_DOCKER_REGISTRY:-ghcr.io}"
     echo "Target Docker registry: $DOCKER_REGISTRY_TARGET"
 
     # Check if already logged in

packages/challenges/agent-challenge/docs/miner/self-deploy.md

@@ -29,7 +29,7 @@ There is no closed agent-model catalog; personal finetunes are banned. Concepts:
 [Attestation TEE — agent-driven order](attestation-tee.md#agent-driven-order-package-verify--tree-sha--tee--eval).
 
 The mission is **CPU Intel TDX only** (no GPU) with a hard **$20** spend cap and a
-preference for the smallest CPU shape that works (`tdx.small`/`tdx.medium`). GPU
+per-stage CPU sizing (review `tdx.small` / 20 GB disk; eval `tdx.xlarge` / 100 GB disk so four concurrent Terminal-Bench tasks fit). GPU
 targets, over-cap shapes, and missing Phala credentials are refused **before** any
 Phala call.
 
@@ -126,7 +126,9 @@ python -m agent_challenge.selfdeploy review deploy \
     --openrouter-key-env OPENROUTER_API_KEY \
     --phala-api https://cloud-api.phala.com/api/v1 \
     --review-instance-type tdx.small \
-    --eval-instance-type tdx.small \
+    --eval-instance-type tdx.xlarge \
+    --review-disk-size-gb 20 \
+    --eval-disk-size-gb 100 \
     --review-runtime-hours 6 \
     --eval-runtime-hours 6 \
     --money-cap-usd 20
@@ -145,7 +147,9 @@ python -m agent_challenge.selfdeploy review deploy \
     --openrouter-key-env OPENROUTER_API_KEY \
     --phala-api https://cloud-api.phala.com/api/v1 \
     --review-instance-type tdx.small \
-    --eval-instance-type tdx.small \
+    --eval-instance-type tdx.xlarge \
+    --review-disk-size-gb 20 \
+    --eval-disk-size-gb 100 \
     --review-runtime-hours 6 \
     --eval-runtime-hours 6 \
     --money-cap-usd 20
@@ -279,6 +283,11 @@ python -m agent_challenge.selfdeploy review retry \
 
 ### `review teardown`
 
+Deletes the review CVM via the Phala Cloud HTTP API (`DELETE /cvms/{id}`).
+No external `phala` CLI binary is required. Pass `--cvm-id` from the deploy
+output, or `--app-id` (the plan `app_identity`) to resolve a unique match from
+`GET /cvms`. Ambiguous matches are refused.
+
 Delete the review CVM after allow, reject, expiry, cancellation, provider
 failure, verification failure, or interruption. The CLI uses the exact
 `phala cvms delete <id> -f` operation. If deletion fails, the command exits
@@ -287,6 +296,8 @@ logs).
 
 ```bash
 python -m agent_challenge.selfdeploy review teardown --cvm-id review-cvm-1
+# or, when the deploy output was lost but app_identity is known:
+python -m agent_challenge.selfdeploy review teardown --app-id <app_identity>
 ```
 
 ### `eval prepare`
@@ -321,6 +332,20 @@ spend projection counts both the review and eval stage shapes against the shared
 money cap. Signing again accepts either `--auto-sign` or explicit `--signature`
 + `--nonce` (+ optional `--timestamp`).
 
+**Host posts the result — the guest only emits.** The measured guest orchestrator
+emits the attested envelope on stdout (a line prefixed `BASE_BENCHMARK_RESULT=`)
+and exits. It never calls `eval result`. The miner host must scrape that line
+and post the exact bytes with the one-time `EVAL_RUN_TOKEN`.
+
+**Live deploy refuses to run without a token handoff.** Pass at least one of:
+
+- `--token-output PATH` (primary) — writes the token with mode `0600`
+- `--emit-run-token` — adds `eval_run_token` to success stdout JSON
+
+Validate the handoff destination **before** prepare spends the single delivery.
+Success stdout always includes `eval_run_id`. The token never appears in
+`--output` plan JSON, redacted prepare/status output, logs, or exception text.
+
 ```bash
 python -m agent_challenge.selfdeploy eval deploy \
     --base-url https://<challenge-host> \
@@ -329,9 +354,25 @@ python -m agent_challenge.selfdeploy eval deploy \
     --auto-sign \
     --llm-cost-limit-env LLM_COST_LIMIT \
     --phala-api https://cloud-api.phala.com/api/v1 \
-    --eval-instance-type tdx.small \
-    --money-cap-usd 20
-```
+    --eval-instance-type tdx.xlarge \
+    --review-disk-size-gb 20 \
+    --eval-disk-size-gb 100 \
+    --money-cap-usd 20 \
+    --token-output ~/.cache/agent-challenge/eval-run.token
+```
+
+Capture `eval_run_id` from deploy stdout (and keep the `0600` token file for
+`eval result`). Optional: `--expected-measurement PATH` compares plan `rtmr0`
+to a local pin JSON before Phala create (mismatch prints truncated prefixes only).
+
+**Shape / measurement-pin footgun.** `--eval-instance-type` must match the
+validator-issued plan `vm_shape` / `instance_type`. A shape change also requires
+a matching `rtmr0` pin on the validator allowlist and a re-prepare. A stale pin
+surfaces only as a generic key-release denial deep in the TEE flow — the CLI
+now aborts before Phala create and names both shapes plus `vm_shape` /
+`instance_type` / `rtmr0`. Prepare already spent the one-shot token on that
+abort path; re-run `eval deploy` (cancel+retry recovery) or explicit
+`eval cancel` + `eval retry` to re-prepare.
 
 Use `--dry-run` to validate the signed plan and show only safe names, digests,
 measurement metadata, and cost. Without a validator eval allowlist the dry-run
@@ -349,6 +390,42 @@ python -m agent_challenge.selfdeploy eval deploy \
 A post-create failure deletes the attributable eval CVM before the command
 returns.
 
+### Host post path after eval CVM finishes
+
+1. Wait for the guest to finish. Scrape guest stdout for the line prefixed
+   `BASE_BENCHMARK_RESULT=` and write the **exact** payload bytes after the
+   prefix into a file (production envelopes are on the order of ~18KB — do not
+   pretty-print or re-encode).
+2. Source the one-time token from the `0600` file without echoing it (no
+   `set -x`):
+
+```bash
+export EVAL_RUN_TOKEN="$(cat ~/.cache/agent-challenge/eval-run.token)"
+# never: echo "$EVAL_RUN_TOKEN" or set -x while the token is in the environment
+```
+
+3. Post with the deploy `eval_run_id`:
+
+```bash
+python -m agent_challenge.selfdeploy eval result \
+    --base-url https://<challenge-host> \
+    --run-id <eval_run_id_from_deploy_stdout> \
+    --result ./eval-result.json \
+    --token-env EVAL_RUN_TOKEN
+```
+
+4. Tear down the eval CVM:
+
+```bash
+python -m agent_challenge.selfdeploy eval teardown --cvm-id <cvm-id>
+# or, when only app_identity is known:
+python -m agent_challenge.selfdeploy eval teardown --app-id <app_identity>
+```
+
+Invariant: `EVAL_RUN_TOKEN` never belongs in plan JSON, compose, ordinary logs,
+or committed files — only the `0600` handoff file and/or one-shot stdout when
+`--emit-run-token` is set.
+
 ### `eval result`
 
 Post the exact CVM-emitted result bytes directly to
@@ -538,7 +615,7 @@ python -m agent_challenge.selfdeploy verdict \
 ### `deploy`
 
 Deploy a CPU-only, miner-funded CVM. Absent `--instance-type`, the smallest CPU
-shape (`tdx.small`) is chosen. A GPU instance type or GPU OS image is refused, and
+shape is stage-specific (review `tdx.small`, eval `tdx.xlarge`). A GPU instance type or GPU OS image is refused, and
 a shape whose projected cost would breach the money cap is refused, both before
 any provisioning. Use `--dry-run` to print the full plan (compose, image digest,
 instance type, region, key-release endpoint, projected cost) and make zero
@@ -613,10 +690,11 @@ python -m agent_challenge.selfdeploy teardown --cvm-id cvm-1
 
 Every CVM you deploy is **miner-funded** and must be deleted when you are done.
 The total mission spend cap is **$20**; always use the smallest CPU shape that
-works (`tdx.small`/`tdx.medium`) and never deploy a GPU CVM. Review and eval
+is stage-sized (review `tdx.small`/20 GB; eval `tdx.xlarge`/100 GB) and never deploy a GPU CVM. Disk is billed separately (~$0.000139/GB/hour) and is included in the projected $20 cap. Review and eval
 spend are projected together before either create.
 
-The `teardown` subcommand runs `phala cvms delete <id> -f` for you, but you can
+The `teardown` subcommand issues `DELETE /cvms/{id}` through the Phala Cloud
+HTTP client (no `phala` binary required). For manual verification you can still
 also delete and confirm directly with the `phala` CLI:
 
 ```

packages/challenges/agent-challenge/docs/validator/self-deploy.md

@@ -274,12 +274,12 @@ emitter.
 
 Any review or eval CVM created for live verification is miner-funded and subject
 to the cumulative review+eval money cap of **$20**. Prefer the smallest CPU shape
-(`tdx.small` / `tdx.medium`) and never a GPU shape. Miners must delete every
+(review `tdx.small`/20 GB disk; eval `tdx.xlarge`/100 GB disk) and never a GPU shape. Disk billing (~$0.000139/GB/hour) is included in the shared $20 projection. Miners must delete every
 attributable CVM after success, reject, expiry, provider failure, quote failure,
 cancellation, interruption, or result failure. Confirm none remain:
 
 ```bash
-phala cvms delete <id> -f
+HTTP `DELETE /cvms/{id}` (or manual `phala cvms delete <id> -f`)
 phala cvms list
 ```
 

packages/challenges/agent-challenge/golden/live-registry-refs.json

@@ -1,25 +1,28 @@
 {
   "schema": "harbor-independence/live-registry-refs@1",
-  "note": "SIDE manifest (NOT the frozen golden dataset digest). Maps a small deterministic subset of Terminal-Bench 2.1 task_ids to PULLABLE, digest-pinned registry refs published to the miner's public Docker Hub namespace so an in-CVM DooD orchestrator can `docker pull` them for a live smoke E2E. This file does NOT affect golden/dataset-digest.json, its per-task content_digest_sha256, the canonical_content_digest_sha256, or the canonical compose/measurement. Resolution is opt-in and fail-closed (see agent_challenge.canonical.live_registry).",
+  "note": "SIDE manifest (NOT the frozen golden dataset digest). Maps a small deterministic subset of Terminal-Bench 2.1 task_ids to PULLABLE, digest-pinned GHCR refs under ghcr.io/baseintelligence so an in-CVM DooD orchestrator can pull them for a live smoke E2E. D6: only GHCR shipping refs are accepted (see agent_challenge.canonical.live_registry). This file does NOT affect golden/dataset-digest.json, its per-task content_digest_sha256, the canonical_content_digest_sha256, or the canonical compose/measurement. Resolution is opt-in and fail-closed. OPS_REQUIRED: orchestrator_image digest is the T1 local buildx manifest (not yet pushed to GHCR); task image digests retain prior content hashes under the new GHCR path pattern and must be republished to GHCR before live pull succeeds.",
   "dataset": "terminal-bench/terminal-bench-2-1",
-  "namespace": "docker.io/mathiiss",
-  "orchestrator_image": "docker.io/mathiiss/agent-challenge-canonical@sha256:02331f0909f617e333f113be376d353770a673669946bcddaac3c53cbde7c9d8",
-  "orchestrator_image_source": "services.yaml build-canonical (`uv run python -m agent_challenge.canonical.build --build`) built reproducibly (BuildKit SOURCE_DATE_EPOCH + rewrite-timestamp, provenance/sbom off) and pushed to the miner Docker Hub namespace",
+  "namespace": "ghcr.io/baseintelligence",
+  "orchestrator_image": "ghcr.io/baseintelligence/agent-challenge-canonical@sha256:ea399cee1b3c9015024918a6070901e6b7b4bee432c8afe1e7dd40a95547e0bc",
+  "orchestrator_image_source": "T1 local agent-recipe buildx (SOURCE_DATE_EPOCH + rewrite-timestamp, provenance/sbom off) → manifest sha256:ea399cee1b3c9015024918a6070901e6b7b4bee432c8afe1e7dd40a95547e0bc; tag was ghcr.io/baseintelligence/agent-challenge-canonical:t1-local. OPS_REQUIRED: replace with the first published GHCR digest from agent-recipe publish-eval-image.yml before T3/prod pin.",
   "tasks": {
     "adaptive-rejection-sampler": {
-      "registry_ref": "docker.io/mathiiss/agent-challenge-tb21-adaptive-rejection-sampler@sha256:7c8bd5835f19506222805de68d65f83d6cca5b502f7d71dc5e2d9d4dd447c0c8",
+      "registry_ref": "ghcr.io/baseintelligence/agent-challenge-tb21-adaptive-rejection-sampler@sha256:7c8bd5835f19506222805de68d65f83d6cca5b502f7d71dc5e2d9d4dd447c0c8",
       "source_ref": "alexgshaw/adaptive-rejection-sampler:20251031",
-      "content_digest_sha256": "bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de"
+      "content_digest_sha256": "bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de",
+      "ops_status": "BLOCKED_REPUBLISH — path retargeted to GHCR; digest is historical content pin until image is pushed under ghcr.io/baseintelligence/"
     },
     "bn-fit-modify": {
-      "registry_ref": "docker.io/mathiiss/agent-challenge-tb21-bn-fit-modify@sha256:c0371862a0861f282eb206471452ea9aa8d494e3687bdc8167c96ab39d73db50",
+      "registry_ref": "ghcr.io/baseintelligence/agent-challenge-tb21-bn-fit-modify@sha256:c0371862a0861f282eb206471452ea9aa8d494e3687bdc8167c96ab39d73db50",
       "source_ref": "alexgshaw/bn-fit-modify:20251031",
-      "content_digest_sha256": "b5f9644970c17ad9ddb46b7266f7bcd87c761d77d7e6f55d7cfe7284d5ff66e9"
+      "content_digest_sha256": "b5f9644970c17ad9ddb46b7266f7bcd87c761d77d7e6f55d7cfe7284d5ff66e9",
+      "ops_status": "BLOCKED_REPUBLISH — path retargeted to GHCR; digest is historical content pin until image is pushed under ghcr.io/baseintelligence/"
     },
     "break-filter-js-from-html": {
-      "registry_ref": "docker.io/mathiiss/agent-challenge-tb21-break-filter-js-from-html@sha256:2a5bd51bab582993befc1a24252c03af673cb1db7cf7d0b2195d66a42a3872a7",
+      "registry_ref": "ghcr.io/baseintelligence/agent-challenge-tb21-break-filter-js-from-html@sha256:2a5bd51bab582993befc1a24252c03af673cb1db7cf7d0b2195d66a42a3872a7",
       "source_ref": "alexgshaw/break-filter-js-from-html:20251031",
-      "content_digest_sha256": "678008d1a4fd1e6e1b9b3cc9a327219fe4b410a31eafc52e9099bbf947eea600"
+      "content_digest_sha256": "678008d1a4fd1e6e1b9b3cc9a327219fe4b410a31eafc52e9099bbf947eea600",
+      "ops_status": "BLOCKED_REPUBLISH — path retargeted to GHCR; digest is historical content pin until image is pushed under ghcr.io/baseintelligence/"
     }
   }
 }

packages/challenges/agent-challenge/src/agent_challenge/canonical/live_registry.py

@@ -4,12 +4,15 @@
 Terminal-Bench task by a *bare content digest* (``harbor_registry_ref =
 "sha256:<64hex>"``) with NO repository, so an in-CVM DooD orchestrator cannot
 ``docker pull`` it. For a live smoke E2E a small deterministic subset of task
-images is published to the miner's public Docker Hub namespace as pullable,
-digest-pinned refs and recorded in a SEPARATE side manifest
+images is published to the org GHCR namespace (``ghcr.io/baseintelligence/…``)
+as pullable, digest-pinned refs and recorded in a SEPARATE side manifest
 (``golden/live-registry-refs.json``). That side manifest never touches
 ``dataset-digest.json`` -- the frozen content digests and the canonical
 measurement stay byte-identical.
 
+D6: shipping live refs MUST be GHCR. ``docker.io`` and the retired
+``mathiiss`` namespace are rejected fail-closed at parse time.
+
 Resolution is **opt-in and fail-closed**: with no live manifest configured
 (no explicit path, no env var) callers get NO live refs and fall back to the
 existing per-task behavior, so flag-off / offline runs are byte-identical. When a
@@ -50,6 +53,9 @@
 # and an un-namespaced ``name@sha256`` so only a real registry ref is accepted.
 _PULLABLE_REF_RE = re.compile(r"^(?=[^@]*/)[A-Za-z0-9][\w.\-/:]*@sha256:[0-9a-f]{64}$")
 
+#: Shipping live-registry refs must live under this registry host prefix (D6).
+LIVE_REGISTRY_HOST_PREFIX = "ghcr.io/"
+
 
 class LiveRegistryError(ValueError):
     """The live-registry side manifest is missing, malformed, or not pullable."""
@@ -83,6 +89,32 @@ def assert_pullable_ref(ref: Any, *, what: str = "registry ref") -> str:
     return ref  # type: ignore[return-value]
 
 
+def assert_live_registry_ref(ref: Any, *, what: str = "registry ref") -> str:
+    """Return ``ref`` if it is a GHCR digest-pinned shipping ref (D6), else raise.
+
+    Builds on :func:`assert_pullable_ref`, then rejects ``docker.io``, the
+    retired ``mathiiss`` namespace, and any non-GHCR host so live smoke never
+    pins a Hub image again.
+    """
+
+    pinned = assert_pullable_ref(ref, what=what)
+    lowered = pinned.lower()
+    if "mathiiss" in lowered:
+        raise LiveRegistryError(
+            f"{what} must not use the retired mathiiss namespace (D6), got {pinned!r}"
+        )
+    if lowered.startswith("docker.io/") or lowered.startswith("index.docker.io/"):
+        raise LiveRegistryError(
+            f"{what} must not use docker.io (D6 — GHCR only), got {pinned!r}"
+        )
+    if not lowered.startswith(LIVE_REGISTRY_HOST_PREFIX):
+        raise LiveRegistryError(
+            f"{what} must be a GHCR digest-pinned ref "
+            f"({LIVE_REGISTRY_HOST_PREFIX}…@sha256:<64hex>), got {pinned!r}"
+        )
+    return pinned
+
+
 @dataclass(frozen=True)
 class LiveRegistry:
     """Parsed live-registry side manifest.
@@ -111,23 +143,24 @@ def __bool__(self) -> bool:
 
 
 def _ref_from_entry(task_id: str, entry: Any) -> str:
-    """Extract + validate the pullable ref from a manifest task entry."""
+    """Extract + validate the pullable GHCR ref from a manifest task entry."""
 
     if isinstance(entry, str):
         ref = entry
     elif isinstance(entry, Mapping):
         ref = entry.get("registry_ref")
     else:
         raise LiveRegistryError(f"live-registry task {task_id!r} is not a string or mapping")
-    return assert_pullable_ref(ref, what=f"live-registry ref for task {task_id!r}")
+    return assert_live_registry_ref(ref, what=f"live-registry ref for task {task_id!r}")
 
 
 def parse_live_registry(data: Mapping[str, Any]) -> LiveRegistry:
     """Parse + validate a live-registry side-manifest document.
 
-    Every task ref must be a pullable ``repo@sha256`` ref (a bare content digest
-    or floating tag is rejected). ``orchestrator_image``, when present, must be
-    pullable too. Task keys are normalized to their bare name.
+    Every task ref must be a GHCR pullable ``repo@sha256`` ref (a bare content
+    digest, floating tag, ``docker.io``, or retired ``mathiiss`` namespace is
+    rejected). ``orchestrator_image``, when present, must be GHCR-pullable too.
+    Task keys are normalized to their bare name.
     """
 
     if not isinstance(data, Mapping):
@@ -143,7 +176,7 @@ def parse_live_registry(data: Mapping[str, Any]) -> LiveRegistry:
 
     orchestrator = data.get("orchestrator_image")
     if orchestrator is not None:
-        orchestrator = assert_pullable_ref(orchestrator, what="orchestrator_image")
+        orchestrator = assert_live_registry_ref(orchestrator, what="orchestrator_image")
 
     return LiveRegistry(orchestrator_image=orchestrator, task_refs=task_refs, raw=dict(data))
 
@@ -203,9 +236,11 @@ def resolve_live_registry_refs(
     "DEFAULT_LIVE_REGISTRY_PATH",
     "LIVE_REGISTRY_ENV",
     "LIVE_REGISTRY_FILENAME",
+    "LIVE_REGISTRY_HOST_PREFIX",
     "LIVE_REGISTRY_SCHEMA",
     "LiveRegistry",
     "LiveRegistryError",
+    "assert_live_registry_ref",
     "assert_pullable_ref",
     "is_pullable_ref",
     "load_live_registry",