From 81da8d743ce38ce356ebc132bb133be359afb018 Mon Sep 17 00:00:00 2001 From: Josef Prochazka Date: Fri, 11 Sep 2026 10:13:19 +0000 Subject: [PATCH 1/3] feat: Give the runtime-specific API its own OpenAPI specification The `/actor-runtime/*` endpoints - live dev folder, debug mode, browser view, migration emulation, upstream API fallback, the per-run events channel - have no counterpart on the Apify platform, so nothing described them except prose scattered through `requirements/api.md`. A client had no way to ask a runtime what it supports. They now have their own OpenAPI 3.1 document, `src/api/openapi/actor-runtime.json`, and that document is what the namespace is: - The runtime serves it from itself, so the CLI can enumerate the namespace in one stock call: `apify api GET /actor-runtime` returns it `{data}`-enveloped, `GET /actor-runtime/openapi.json` returns it bare for OpenAPI tooling. Both are unauthenticated, and the real platform answers `404` on the same path, so the call doubles as "am I pointed at a local runtime, and which one?". - Everything under `/actor-runtime/*` the document does not describe is now answered from the document rather than falling through to the catch-all that only knows the emulated platform surface: an undescribed path is `404 not-found` pointing back at `GET /actor-runtime`, a described path with an undescribed method is `405 method-not-allowed` with an `Allow` header, and a plain request to the events websocket path is `426 upgrade-required` instead of a bare `401`/`404`. - `requirements/api.md` now references the document for paths, bodies, payloads and per-rejection error types, and keeps only what OpenAPI cannot express: behaviour over time, cross-surface consistency, and the fallback/migration guarantees. An integration test drives every documented operation, so a route without a document entry loses its 404/405 contract and a document entry without a route answers `501` - neither can drift silently. Closes #40 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016VvUV6661cyYbxoHZv2Pby --- CLAUDE.MD | 5 + README.md | 18 + requirements/api.md | 150 +++--- requirements/cli.md | 13 + src/api/actor-runtime-spec.ts | 115 +++++ src/api/openapi/actor-runtime.json | 529 ++++++++++++++++++++ src/api/routes/actor-runtime-spec.ts | 107 ++++ src/api/server.ts | 11 + test/integration/actor-runtime-spec.test.ts | 101 ++++ test/integration/api-fallback.test.ts | 10 +- test/unit/actor-runtime-spec.test.ts | 147 ++++++ 11 files changed, 1113 insertions(+), 93 deletions(-) create mode 100644 src/api/actor-runtime-spec.ts create mode 100644 src/api/openapi/actor-runtime.json create mode 100644 src/api/routes/actor-runtime-spec.ts create mode 100644 test/integration/actor-runtime-spec.test.ts create mode 100644 test/unit/actor-runtime-spec.test.ts diff --git a/CLAUDE.MD b/CLAUDE.MD index e054f0b0..22fa31b4 100644 --- a/CLAUDE.MD +++ b/CLAUDE.MD @@ -23,6 +23,11 @@ Local Actor runtime is an Actor development tool for developing, running, and de - To redirect Apify CLI back to the original Apify services unset the environment variables: - `APIFY_CLIENT_BASE_URL` - `APIFY_CONSOLE_URL` +- To see everything this runtime adds on top of the Apify API (the `/actor-runtime/*` endpoints used + below), read its own OpenAPI specification: `apify api GET /actor-runtime` (no token needed; the real + Apify platform has no such endpoint, so a `404` there means you are talking to the platform, not to a + local runtime). `http://localhost:3333/actor-runtime/openapi.json` serves the same document + unenveloped for OpenAPI tooling. - Use `apify cli api ...` to send API calls and inspect the Actors, builds, runs, storages and other objects. - To simulate multiple users use custom token and in additional authorization header. For example: `apify cli api v2/datasets -H '{"authorization": "Bearer TOKEN"}'` - You can use already authenticated CLI or call `apify login --token TOKEN` diff --git a/README.md b/README.md index ce188e86..ba50d571 100644 --- a/README.md +++ b/README.md @@ -99,6 +99,24 @@ Good to know: - `podman images` lists the images the runtime builds as `actor-runtime/:` under the registry prefix Podman adds itself (`docker.io/` or `localhost/`, depending on the version). +## What this runtime adds on top of the Apify API + +Everything under `/actor-runtime/*` is local-runtime-only - the dev folder, debug mode, browser view, +migration emulation and the upstream API fallback used in the sections below. The runtime describes +that namespace to itself in an OpenAPI document (`src/api/openapi/actor-runtime.json`) and serves it, +so you never have to guess what a given runtime supports: + +```bash +apify api GET /actor-runtime # every runtime-specific endpoint, its body and its responses +curl -s http://localhost:3333/actor-runtime/openapi.json | jq .paths # same document, for tooling +``` + +Both are unauthenticated, and the real Apify platform has no such endpoint - so the same call also +answers "am I pointed at a local runtime or at the platform?". Anything under `/actor-runtime/*` the +document does not describe is rejected from the document too: an unknown path is a `404` pointing you +back at `GET /actor-runtime`, and a known path with the wrong method is a `405` naming the methods it +does have. + ## Rapid dev loop: bind-mounting your local source (no rebuild per edit) After the one push+build above, register your Actor's local source folder so every future run picks up diff --git a/requirements/api.md b/requirements/api.md index bc6f6a60..6c8cf52e 100644 --- a/requirements/api.md +++ b/requirements/api.md @@ -17,7 +17,7 @@ - `DELETE /v2/actor-builds/:buildId` and `DELETE /v2/actor-runs/:runId` on a **non-terminal** build/run are rejected, not aborted-then-deleted: `400` with error type `deleting-unfinished-build` (builds) or `cannot-remove-running-run` (runs), matching the Apify platform. -- Three endpoints are exceptions to the `{data}` envelope: +- Four endpoints are exceptions to the `{data}` envelope: - `GET /v2/logs/:buildOrRunId` (and its `actor-builds`/`actor-runs` aliases): the body is plain text, never `{data}`-wrapped, matching apify-client-js's `log().get()`. - `GET /v2/datasets/:datasetId/items` (and its `actor-runs/:runId/dataset/items` alias): the body is @@ -25,6 +25,9 @@ `x-apify-pagination-*` response headers, matching apify-client-js's pagination handling. - `GET /actor-runtime/events/:runId`: a websocket upgrade, not a JSON response at all - see "Actor runtime API" below. + - `GET /actor-runtime/openapi.json`: the runtime's own OpenAPI document, served bare so standard + OpenAPI tooling can consume the URL - see "Actor runtime API" below. The same document _is_ + `{data}`-enveloped at `GET /actor-runtime`, which is what the CLI reads. - `*At` timestamp fields are ISO-8601 strings. - Log content matches the Apify platform's log format: every log line starts with an ISO-8601 UTC timestamp with millisecond precision followed by a space (`2026-08-31T09:13:25.123Z `), exactly one @@ -39,6 +42,8 @@ # 501 vs 404 +- This section is about the emulated platform surface only. The `/actor-runtime/*` namespace has its + own specification and its own rule for what it does not describe - see "Actor runtime API" below. - Which endpoints answer `501` (unimplemented spec path) instead of `404` (off-spec path entirely) is decided from a fixed, built-in list of known Apify API v2 spec paths - nothing is fetched from `docs.apify.com` at runtime. See "Known differences from the Apify platform" in `storage.md` for the @@ -120,82 +125,59 @@ read/write/delete) just above, which this runtime does implement. Both paths answer `501`, not `404`. - All endpoints from the specification that do not have implementation must return response `501 Not Implemented` -- All endpoints not present in specification must return `404 Not Found` - **except** the `/actor-runtime/*` +- All endpoints not present in specification must return `404 Not Found` - **except** `/actor-runtime/*`, + which is not part of the Apify API at all and answers from its own specification instead ("Actor + runtime API" below) # Actor runtime API -- `/actor-runtime/*` is the API controlling functions specific to the local Actor runtime -- **`POST /actor-runtime/dev-folder/:actorId`** - registers (or clears) the Actor's local dev folder for - the bind-mount feature (`actor-driver.md`). `:actorId` accepts the same forms as the rest of the API - (id, plain name, `username~name`). - - **Authenticated** the same way as every `/v2` route, and scoped to the caller's own Actors. - - **No build-first precondition** - registration works for an Actor that has never been built at all. - - **Request body**: a JSON string - the absolute path to set, or `""` to clear. - - **Response**: on success, `{ data: { localDevFolder } }` - the same value the console detail page - shows (`console.md`), doubling as the read-back this design has no separate `GET` for. - - **Error responses**, by rejection reason: - - `400` `invalid-request` - the body isn't a JSON string, or the string isn't a valid absolute - path. - - `400` `dev-folder-path-not-found` - the path does not exist on the host. - - `400` `dev-folder-not-a-directory` - the path exists but is not a directory. - - `400` `dev-folder-check-failed` - the path could not be verified, for any other reason. - - `503` `dev-folder-check-unavailable` - Docker itself is unreachable. - - `500` `internal-error` - an operational fault unrelated to the submitted path. -- The console's own dev-folder form (`console.md`) does **not** go through this endpoint - it posts to a - console-local, unauthenticated route on the console's own port - but the two surfaces accept and - reject exactly the same inputs with the same outcomes. +- `/actor-runtime/*` is the API controlling functions specific to the local Actor runtime: developer + conveniences (live dev folder, debug mode, browser view, migration emulation, upstream API fallback, + the per-run events channel) that the real Apify platform API has no counterpart for. +- **The namespace has its own OpenAPI specification**, committed at `src/api/openapi/actor-runtime.json`. + That document is the normative contract for every endpoint in it - paths, methods, request bodies, + response payloads, per-rejection error `type`s, and worked examples. This file does not repeat it: + the sections below state only what OpenAPI cannot express (behaviour over time, cross-surface + consistency, and the guarantees the fallback and migration features rest on). +- **The runtime serves that specification from itself**, so a client can enumerate what a given + runtime supports rather than hard-coding a list: + - **`GET /actor-runtime`** - the document in the usual `{data}` envelope, so it reads through + apify-client-js and therefore through `apify api GET /actor-runtime` (`cli.md`). + - **`GET /actor-runtime/openapi.json`** - the same document unenveloped, for OpenAPI tooling + pointed straight at the URL. + - Both are **unauthenticated**, unlike every other endpoint in the namespace: the document is + static, identical for every caller and carries no user data, so a client can identify a local + Actor runtime and enumerate its capabilities before it holds a token. + - The document's `info.version` is the runtime's own version. +- Every endpoint in the namespace is served at both `/actor-runtime/*` (canonical) and + `/v2/actor-runtime/*` (the same routes, reachable a second way purely because `apify api` builds + every URL against a base that already ends in `/v2`). Neither mount is part of the emulated Apify + API, and neither is ever relayed upstream (see "Upstream fallback" below). +- Every endpoint except the two specification endpoints above and the events websocket (below) is + **authenticated** the same way as every `/v2` route and **scoped to the caller's own** Actors/runs, + and none has a **build-first precondition** - a toggle can be set for an Actor that has never been built at all. The endpoints + that set a per-Actor toggle (dev folder, debug mode, browser view) have no separate `GET`: each + response body doubles as the read-back, and each call fully replaces the prior state rather than + merging into it. +- **Anything under `/actor-runtime/*` that the specification does not describe is answered from the + specification**, never from the emulated platform surface: + - an undescribed path answers `404` `not-found`, with a message pointing at `GET /actor-runtime`; + - a described path addressed with an undescribed method answers `405` `method-not-allowed` with an + `Allow` header naming the methods it does have; + - a plain HTTP request to the events websocket path answers `426` `upgrade-required`. +- The console's own dev-folder, debug-mode and browser-view forms (`console.md`) do **not** go through + these endpoints - they post to console-local, unauthenticated routes on the console's own port - but + the two surfaces accept and reject exactly the same inputs with the same outcomes. - **`POST /v2/actors/:actorId/runs?devFolder=false`** - runs from the built image alone, ignoring the - registered dev folder for that one run only; the registration itself is unchanged. Any other value, or - no parameter, means the default behaviour. -- **`POST /actor-runtime/debug/:actorId`** - sets (or clears) the Actor's persistent debug-mode toggle - (`actor-driver.md`'s "Debug mode" section). `:actorId` accepts the same forms as the rest of the API. - - **Authenticated** the same way as every `/v2` route, and scoped to the caller's own Actors. - - **No build-first precondition** - the toggle itself needs no build to exist. - - **Request body**: a strict JSON object with exactly these fields: - - `enabled` (required, boolean). - - `language` (optional, one of `"auto"` / `"node"` / `"python"`; defaults to `"auto"`). - - `port` (optional, integer `1024..65535`; absent means "use the resolved language's own - default port at run start" - never a stored literal). - Any other key present is rejected. Every accepted call fully replaces the prior state for that - Actor (never a partial merge) - a field the body omits resets to its own default, it does not keep - whatever a previous call set. `{"enabled": false}` clears the whole toggle, whatever else the body - names. - - **Response**: on success, `{ data: { localDebug } }`, where `localDebug` is `null` when debug mode - is off, or `{ language, port }` when on - `port` here is a nominal default (`5678`) for an - unresolved `language: "auto"`, purely for display; the port a given run actually publishes depends - on that run's own resolved language (`actor-driver.md`). Same doubles-as-read-back contract as the - dev-folder endpoint - no separate `GET`. - - **Error responses**: `400` `invalid-request` for every malformed body (not a JSON object, an - unknown field, a missing/non-boolean `enabled`, an invalid `language`, or a `port` outside - `1024..65535`) - no state change on rejection. - - Worked examples: - ``` - POST /actor-runtime/debug/ --body '{"enabled": true}' - -> { "data": { "localDebug": { "language": "auto", "port": 5678 } } } - - POST /actor-runtime/debug/ --body '{"enabled": true, "language": "node", "port": 9229}' - -> { "data": { "localDebug": { "language": "node", "port": 9229 } } } - - POST /actor-runtime/debug/ --body '{"enabled": false}' - -> { "data": { "localDebug": null } } - - POST /actor-runtime/debug/ --body '{"enabled": true, "prot": 9229}' - -> 400 invalid-request "Unknown field \"prot\" - allowed fields are \"enabled\", \"language\", \"port\"." - ``` - - The console's own debug-mode form (`console.md`) does **not** go through this endpoint - same - console-local, unauthenticated split as the dev-folder form - but both surfaces accept and reject - exactly the same inputs with the same outcomes. -- **`POST /actor-runtime/browser-view/:actorId`** - sets or clears the Actor's browser-view toggle - (`actor-driver.md`'s "Browser view" section). Authenticated and owner-scoped like every `/v2` route; no - build-first precondition. - - **Body**: `{ "enabled": boolean, "interactive"?: boolean }`, `interactive` defaulting to `false`. A call - fully replaces the prior state; `{"enabled": false}` clears it. Any other shape is `400 invalid-request`. - - **Response**: `{ data: { localBrowserView: { interactive } | null } }` - the read-back; there is no `GET`. -- **`GET /actor-runtime/events/:runId`** - a websocket upgrade, reachable at exactly this one path on - the fixed API port (`system.md`). It carries the run's platform events: `systemInfo` once a second - (`actor-driver.md`), a one-off `aborting`-plus-`persistState` pair under `?gracefully=` (below), and a - one-off `migrating` frame when a migration is triggered ("Migration emulation" below). Each frame is a - single text message, `{"name": "...", "data": {...}}`. + registered dev folder for that one run only; the registration itself is unchanged. Any other value, + or no parameter, means the default behaviour. A runtime-only query parameter on an otherwise + faithful platform endpoint, so it lives on the platform surface rather than in this namespace; the + specification lists it under `x-actor-runtime-platform-extensions` so a client enumerating the + document still sees it. +- **The events websocket** (`GET /actor-runtime/events/:runId`) carries the run's platform events: + `systemInfo` once a second (`actor-driver.md`), a one-off `aborting`-plus-`persistState` pair under + `?gracefully=` (below), and a one-off `migrating` frame when a migration is triggered ("Migration + emulation" below). It is reachable at exactly this one path on the fixed API port (`system.md`). - The endpoint has no authentication. The run id in the path is the only thing it scopes on, and a connection only ever receives that run's own frames; one run never sees another's. - An unknown or already-terminal run id gets a completed upgrade followed immediately by a `1008` @@ -236,8 +218,6 @@ This runtime emulates that observable experience on demand: the container env are unchanged. `stats.migrationCount` increments once per performed stop. - Responds immediately with the run object (same shape as `abort`/`reboot`). A second call during the open window joins it: same response, no second frame or window. - - Errors: unknown/foreign run `404` `record-not-found`; finished run `403` `job-finished`; - `READY`/`ABORTING` `400` `invalid-request`. - The timeout budget is per run, not per container: a restarted container gets only the remaining `timeoutSecs`. - An abort (graceful or hard) landing during the window or restart wins: the run ends `ABORTED`, @@ -256,20 +236,12 @@ This runtime emulates that observable experience on demand: a request this runtime cannot satisfy locally is instead relayed to the real Apify platform. Both default to `false`, and a restart always brings both back to `false`, regardless of how they were last set. Either can be on without the other; all four combinations are valid. -- **`GET /actor-runtime/api-fallback`** (also reachable at `/v2/actor-runtime/api-fallback`, like every - other endpoint in this namespace) returns - `{ "data": { "fallbackUnimplementedEnabled": , "fallbackNotFoundEnabled": , "upstreamBaseUrl": } }`. - `upstreamBaseUrl` is the platform this runtime would relay to (default `https://api.apify.com`, or the - value of `APIFY_UPSTREAM_API_BASE_URL` if set) - reported for visibility, but read-only: no request - body can change it. -- **`POST /actor-runtime/api-fallback`** (same two mounts) accepts a body naming either field, or both; - a field the body doesn't mention keeps its current value. The response is the same shape `GET` - returns, showing the state immediately after the change. - - **Authenticated** the same way as every other route in this namespace: no token is `401` - `user-not-authenticated`, with no state change. - - **Error responses**: a body that isn't a JSON object (a JSON array, scalar, or `null`), a body - present but empty (`{}`), a body containing a key other than the two above, or a body where a - present key's value isn't a boolean, is `400` `invalid-request`, with no state change. +- **`GET`/`POST /actor-runtime/api-fallback`** read and change that state; the request and response + shapes are in the specification. `POST` is a partial update - a field the body doesn't mention keeps + its current value - and is the only way to change the state: `upstreamBaseUrl` is reported on every + response for visibility but is read-only (it is the platform this runtime would relay to, + `https://api.apify.com` by default, or the value of `APIFY_UPSTREAM_API_BASE_URL`). A rejected body + changes nothing, not even the fields that would have passed on their own. - **Which local outcome each toggle covers** (exhaustive - every other error response is never eligible, under any toggle combination): - `fallbackUnimplementedEnabled` covers a request the runtime does not serve at all: a local `404` diff --git a/requirements/cli.md b/requirements/cli.md index e1ef41b5..1937084a 100644 --- a/requirements/cli.md +++ b/requirements/cli.md @@ -43,6 +43,19 @@ With neither source, `/users/me`'s `proxy` field is omitted and no `APIFY_PROXY_PASSWORD` is set on run containers - never a placeholder. +## Enumerating the runtime's own endpoints + +- The runtime-specific endpoints (`/actor-runtime/*`, `api.md`) have no counterpart on the Apify + platform, so a client cannot learn them from the platform's own OpenAPI specification. The runtime + therefore serves its own specification for that namespace, and the CLI reads it with a single stock + call: **`apify api GET /actor-runtime`** - one response enumerating every runtime-specific endpoint, + its request body, and its responses. +- No token is needed for that call, and the real Apify platform answers `404` on the same path, so it + doubles as "is the CLI pointed at a local Actor runtime, and which one?" - the document's + `info.version` is the runtime's own version. +- A client that asks for something in that namespace which the specification does not describe is told + so in those terms: `404` naming `GET /actor-runtime`, or `405` with an `Allow` header (`api.md`). + ## Supported commands (POC) - `apify push` - creates the Actor and Actor version from local source and triggers diff --git a/src/api/actor-runtime-spec.ts b/src/api/actor-runtime-spec.ts new file mode 100644 index 00000000..4ed5b4b1 --- /dev/null +++ b/src/api/actor-runtime-spec.ts @@ -0,0 +1,115 @@ +/** + * The `/actor-runtime/*` namespace's own OpenAPI document (`openapi/actor-runtime.json`) and the + * lookups the server drives from it. + * + * The document - not this module, and not `requirements/api.md` - is the single source of truth for + * which runtime-specific endpoints exist: `routes/actor-runtime-spec.ts` serves it verbatim so a client + * can enumerate the namespace, and the namespace's terminal handler decides `404` vs `405` vs `426` + * from these lookups. Adding an endpoint therefore means adding it to the document; a test + * (`test/integration/actor-runtime-spec.test.ts`) fails if a documented HTTP operation has no route + * behind it, so the two cannot drift apart silently. + * + * This is the namespace-local counterpart of `spec-table.ts`, which does the same 501-vs-404 job for + * the emulated Apify `/v2` surface from a vendored snapshot of the platform's own spec. + */ +import rawDocument from './openapi/actor-runtime.json' with { type: 'json' }; + +/** Only the parts of OpenAPI this runtime actually reads - not a general-purpose OpenAPI model. */ +interface OperationObject { + operationId?: string; + summary?: string; + /** `"websocket"` marks an operation served by the HTTP upgrade handler rather than an Express route. */ + 'x-actor-runtime-transport'?: string; +} + +interface OpenApiDocument { + openapi: string; + info: { title: string; version: string; description?: string }; + paths: Record>; +} + +/** Lower-case, as OpenAPI spells them; anything else under a path item (`parameters`, `summary`, ...) + * is not an operation. */ +const HTTP_METHODS = ['get', 'put', 'post', 'delete', 'options', 'head', 'patch', 'trace'] as const; + +export const ACTOR_RUNTIME_OPENAPI = rawDocument as unknown as OpenApiDocument; + +export interface ActorRuntimeOperation { + /** Upper-case, matching `req.method`. */ + method: string; + /** The OpenAPI path template, e.g. `/actor-runtime/debug/{actorId}`. */ + path: string; + operationId: string; + summary: string; + /** `websocket` operations are upgraded by `events-ws.ts`, so no Express route ever matches them. */ + transport: 'http' | 'websocket'; +} + +function readOperations(document: OpenApiDocument): ActorRuntimeOperation[] { + const operations: ActorRuntimeOperation[] = []; + for (const [path, pathItem] of Object.entries(document.paths)) { + for (const method of HTTP_METHODS) { + const operation = pathItem[method] as OperationObject | undefined; + if (!operation) continue; + operations.push({ + method: method.toUpperCase(), + path, + operationId: operation.operationId ?? `${method}${path}`, + summary: operation.summary ?? '', + transport: operation['x-actor-runtime-transport'] === 'websocket' ? 'websocket' : 'http', + }); + } + } + return operations; +} + +export const ACTOR_RUNTIME_OPERATIONS: readonly ActorRuntimeOperation[] = readOperations(ACTOR_RUNTIME_OPENAPI); + +/** `/actor-runtime/events/` and `/actor-runtime/events` are the same path here; the empty trailing + * segment is not a segment. A bare `/actor-runtime` normalizes to no segments beyond the namespace. */ +function segmentsOf(path: string): string[] { + return path.split('/').filter(Boolean); +} + +/** Structural match, same rule as `spec-table.ts`: equal segment count, literal segments equal, + * `{param}` segments wildcard. */ +function pathMatches(template: string, requestPath: string): boolean { + const templateSegments = segmentsOf(template); + const requestSegments = segmentsOf(requestPath); + if (templateSegments.length !== requestSegments.length) return false; + return templateSegments.every( + (segment, i) => (segment.startsWith('{') && segment.endsWith('}')) || segment === requestSegments[i], + ); +} + +/** + * Every operation the document describes on the path template `requestPath` matches - empty when the + * document describes no such path at all. The namespace's terminal handler branches on this: empty is + * a `404` (off-spec path), non-empty with no matching method is a `405` (the `Allow` header is built + * from exactly these). + * + * `requestPath` is the full namespace path (`/actor-runtime/...`), with or without a trailing slash, + * and without a query string. + */ +export function actorRuntimeOperationsAtPath(requestPath: string): ActorRuntimeOperation[] { + return ACTOR_RUNTIME_OPERATIONS.filter((operation) => pathMatches(operation.path, requestPath)); +} + +/** + * The router-relative Express path for a documented operation - `/actor-runtime/events/{runId}` -> + * `/events/:runId` - so a route can be registered from the document instead of from a second, + * hand-maintained copy of the same path. + */ +export function routerPathOf(operation: ActorRuntimeOperation): string { + const relative = segmentsOf(operation.path) + .slice(1) + .map((segment) => (segment.startsWith('{') && segment.endsWith('}') ? `:${segment.slice(1, -1)}` : segment)) + .join('/'); + return `/${relative}`; +} + +/** The one operation the document describes for this method and path, if any. */ +export function matchActorRuntimeOperation(method: string, requestPath: string): ActorRuntimeOperation | undefined { + const upperMethod = method.toUpperCase(); + return actorRuntimeOperationsAtPath(requestPath).find((operation) => operation.method === upperMethod); +} diff --git a/src/api/openapi/actor-runtime.json b/src/api/openapi/actor-runtime.json new file mode 100644 index 00000000..deaa392a --- /dev/null +++ b/src/api/openapi/actor-runtime.json @@ -0,0 +1,529 @@ +{ + "openapi": "3.1.0", + "info": { + "title": "Apify local Actor runtime API", + "version": "0.1.0", + "summary": "The local-runtime-only endpoints that have no counterpart on the Apify platform.", + "description": "Everything under `/actor-runtime/*` is specific to this local Actor runtime: developer-convenience controls (live dev folder, debug mode, browser view, migration emulation, upstream API fallback) that the real Apify platform API does not have. The emulated subset of the platform API itself is *not* described here - that surface follows https://docs.apify.com/api/openapi.json.\n\nThis document is the normative contract for the namespace; `requirements/api.md` references it rather than repeating it. The runtime serves it from itself, so a client can discover what a given runtime supports: `GET /actor-runtime` returns it in the API's `{data}` envelope (readable through `apify api GET /actor-runtime`), `GET /actor-runtime/openapi.json` returns the bare document for OpenAPI tooling.\n\nA request under `/actor-runtime/*` that this document does not describe never reaches the emulated platform API and is never relayed upstream: an undescribed path answers `404 not-found`, a described path addressed with an undescribed method answers `405 method-not-allowed` with an `Allow` header.", + "license": { "name": "Apache-2.0", "identifier": "Apache-2.0" } + }, + "servers": [ + { + "url": "/", + "description": "Canonical mount on the runtime's API port (3333 by default, fixed - see requirements/system.md)." + }, + { + "url": "/v2", + "description": "Alias mount serving the exact same routes, so `apify api /actor-runtime/...` reaches them: the CLI builds every URL against a base that already ends in `/v2`." + } + ], + "security": [{ "bearerAuth": [] }, { "tokenQuery": [] }], + "x-actor-runtime-platform-extensions": [ + { + "method": "POST", + "path": "/v2/actors/{actorId}/runs", + "parameter": "devFolder", + "description": "`?devFolder=false` runs from the built image alone, ignoring the Actor's registered live dev folder for that one run; the registration itself is unchanged. Any other value, or no parameter, means the default behaviour. A runtime-only query parameter on an otherwise faithful platform endpoint, listed here so a client enumerating this document sees every local deviation in one place." + } + ], + "paths": { + "/actor-runtime": { + "get": { + "operationId": "getActorRuntimeSpecification", + "summary": "Enumerate the runtime-specific API", + "description": "Returns this OpenAPI document, `{data}`-enveloped like every other JSON response on this API, so `apify api GET /actor-runtime` prints it. Unauthenticated: the document is static, identical for every caller, and carries no user data - a client can therefore identify a local Actor runtime and enumerate its runtime-specific endpoints before it has a token.", + "security": [], + "responses": { + "200": { + "description": "The runtime-specific OpenAPI document.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { "data": { "$ref": "#/components/schemas/OpenApiDocument" } } + } + } + } + } + } + } + }, + "/actor-runtime/openapi.json": { + "get": { + "operationId": "getActorRuntimeOpenApiDocument", + "summary": "The bare OpenAPI document", + "description": "The same document as `GET /actor-runtime`, served without the `{data}` envelope so standard OpenAPI tooling can consume the URL directly. One of the documented exceptions to the envelope rule in `requirements/api.md`. Unauthenticated for the same reason.", + "security": [], + "responses": { + "200": { + "description": "The runtime-specific OpenAPI document, unenveloped.", + "content": { + "application/json": { "schema": { "$ref": "#/components/schemas/OpenApiDocument" } } + } + } + } + } + }, + "/actor-runtime/dev-folder/{actorId}": { + "post": { + "operationId": "setActorDevFolder", + "summary": "Register or clear an Actor's live dev folder", + "description": "Registers the host directory bind-mounted over the image's working directory on every subsequent run of this Actor, so source edits take effect without a rebuild (`requirements/actor-driver.md`). Submitting `\"\"` clears the registration.\n\nScoped to the caller's own Actors. There is no build-first precondition: registration works for an Actor that has never been built. The response body doubles as the read-back - there is deliberately no separate `GET`.\n\nThe console's own dev-folder form is a console-local, unauthenticated route on the console's port and does not go through this endpoint, but both surfaces accept and reject exactly the same inputs with the same outcomes.", + "parameters": [{ "$ref": "#/components/parameters/ActorId" }], + "requestBody": { + "required": true, + "description": "A JSON string: the absolute path to register, or `\"\"` to clear.", + "content": { + "application/json": { + "schema": { "type": "string" }, + "examples": { + "register": { "summary": "Register a folder", "value": "/abs/path/to/src" }, + "clear": { "summary": "Clear the registration", "value": "" } + } + } + } + }, + "responses": { + "200": { + "description": "The registration after the change.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { "data": { "$ref": "#/components/schemas/DevFolderStatus" } } + } + } + } + }, + "400": { + "description": "Rejected, with no state change. Error `type` names the reason: `invalid-request` (the body is not a JSON string, or the string is not a valid absolute path), `dev-folder-path-not-found` (the path does not exist on the host), `dev-folder-not-a-directory` (the path exists but is not a directory), `dev-folder-check-failed` (the path could not be verified, for any other reason).", + "content": { + "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + } + }, + "401": { "$ref": "#/components/responses/Unauthenticated" }, + "404": { "$ref": "#/components/responses/RecordNotFound" }, + "500": { + "description": "`internal-error` - an operational fault unrelated to the submitted path.", + "content": { + "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + } + }, + "503": { + "description": "`dev-folder-check-unavailable` - Docker itself is unreachable, so the path could not be checked.", + "content": { + "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + } + } + } + } + }, + "/actor-runtime/debug/{actorId}": { + "post": { + "operationId": "setActorDebugMode", + "summary": "Set or clear an Actor's debug-mode toggle", + "description": "While debug mode is on, every run of this Actor starts paused waiting for a debugger, with the attach address printed in the run's own log and the debug port published on `127.0.0.1` (`requirements/actor-driver.md`).\n\nScoped to the caller's own Actors; no build-first precondition. Every accepted call fully replaces the prior state - never a partial merge, so a field the body omits resets to its own default rather than keeping what a previous call set. `{\"enabled\": false}` clears the whole toggle, whatever else the body names. The response doubles as the read-back; there is no separate `GET`.\n\nThe console's own debug-mode form is a console-local, unauthenticated route and does not go through this endpoint, but both surfaces accept and reject exactly the same inputs with the same outcomes.", + "parameters": [{ "$ref": "#/components/parameters/ActorId" }], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { "$ref": "#/components/schemas/DebugModeRequest" }, + "examples": { + "enableAutoDetected": { + "summary": "Turn on, auto-detect the language", + "value": { "enabled": true } + }, + "enableNode": { + "summary": "Turn on for Node on an explicit port", + "value": { "enabled": true, "language": "node", "port": 9229 } + }, + "disable": { "summary": "Turn off", "value": { "enabled": false } } + } + } + } + }, + "responses": { + "200": { + "description": "The toggle after the change.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { "data": { "$ref": "#/components/schemas/DebugStatus" } } + }, + "examples": { + "enabledAutoDetected": { + "value": { "data": { "localDebug": { "language": "auto", "port": 5678 } } } + }, + "enabledNode": { + "value": { "data": { "localDebug": { "language": "node", "port": 9229 } } } + }, + "disabled": { "value": { "data": { "localDebug": null } } } + } + } + } + }, + "400": { + "description": "`invalid-request`, with no state change: the body is not a JSON object, names an unknown field, misses `enabled` or gives it a non-boolean, sets an invalid `language`, or sets a `port` outside `1024..65535`. Example message for an unknown field: `Unknown field \"prot\" - allowed fields are \"enabled\", \"language\", \"port\".`", + "content": { + "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + } + }, + "401": { "$ref": "#/components/responses/Unauthenticated" }, + "404": { "$ref": "#/components/responses/RecordNotFound" } + } + } + }, + "/actor-runtime/browser-view/{actorId}": { + "post": { + "operationId": "setActorBrowserView", + "summary": "Set or clear an Actor's browser-view toggle", + "description": "While browser view is on, every run of this Actor prints a viewer URL in its log - a live view of the display the Actor's browser draws on, served on the console's port (`requirements/actor-driver.md`). `interactive` also forwards mouse and keyboard input.\n\nScoped to the caller's own Actors; no build-first precondition. A call fully replaces the prior state; `{\"enabled\": false}` clears it. The response doubles as the read-back; there is no separate `GET`.", + "parameters": [{ "$ref": "#/components/parameters/ActorId" }], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { "$ref": "#/components/schemas/BrowserViewRequest" }, + "examples": { + "watch": { "summary": "Watch only", "value": { "enabled": true } }, + "interact": { + "summary": "Watch and control", + "value": { "enabled": true, "interactive": true } + }, + "disable": { "summary": "Turn off", "value": { "enabled": false } } + } + } + } + }, + "responses": { + "200": { + "description": "The toggle after the change.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { "data": { "$ref": "#/components/schemas/BrowserViewStatus" } } + } + } + } + }, + "400": { + "description": "`invalid-request`, with no state change - any body other than `{\"enabled\": boolean, \"interactive\"?: boolean}`.", + "content": { + "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + } + }, + "401": { "$ref": "#/components/responses/Unauthenticated" }, + "404": { "$ref": "#/components/responses/RecordNotFound" } + } + } + }, + "/actor-runtime/migrate/{runId}": { + "post": { + "operationId": "migrateRun", + "summary": "Emulate a platform migration of a running run", + "description": "Gives the run the platform's migration experience: a `migrating` frame on its events channel immediately, its container stopped a few seconds later, then a fresh container for the same run - same run id, env vars and default storages, in-memory state gone. The run never leaves `RUNNING`, and `stats.migrationCount` increments once per performed stop.\n\nScoped to the caller's own runs. Responds immediately with the run object, the same shape `abort`/`reboot` return; a second call during the open window joins it - same response, no second frame or window. The console's run detail view exposes the same trigger as a Migrate button.", + "parameters": [{ "$ref": "#/components/parameters/RunId" }], + "responses": { + "200": { + "description": "The run object, read back after the migration was scheduled.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { "data": { "$ref": "#/components/schemas/Run" } } + } + } + } + }, + "400": { + "description": "`invalid-request` - the run is non-terminal but has no container to migrate (`READY` or `ABORTING`).", + "content": { + "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + } + }, + "401": { "$ref": "#/components/responses/Unauthenticated" }, + "403": { + "description": "`job-finished` - the run has already finished.", + "content": { + "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + } + }, + "404": { "$ref": "#/components/responses/RecordNotFound" } + } + } + }, + "/actor-runtime/api-fallback": { + "get": { + "operationId": "getApiFallbackState", + "summary": "Read the upstream API fallback state", + "responses": { + "200": { "$ref": "#/components/responses/ApiFallbackState" }, + "401": { "$ref": "#/components/responses/Unauthenticated" } + } + }, + "post": { + "operationId": "setApiFallbackState", + "summary": "Turn upstream API fallback on or off", + "description": "Two independent toggles gate whether a request this runtime cannot satisfy locally is relayed to the real Apify platform instead of failing: `fallbackUnimplementedEnabled` covers a local `404`/`501` (nothing local answers this), `fallbackNotFoundEnabled` covers a `record-not-found` on a route this runtime does serve. Both default to `false` and a restart always brings them back to `false`.\n\nA field the body does not mention keeps its current value. All HTTP methods are eligible for relaying, writes included, and a relayed request carries the caller's own token to the upstream platform - see `requirements/api.md`'s \"Upstream fallback\" section for the full eligibility, fail-closed and token-forwarding contract.", + "requestBody": { + "required": true, + "description": "A JSON object naming either toggle, or both.", + "content": { + "application/json": { + "schema": { "$ref": "#/components/schemas/ApiFallbackPatch" }, + "examples": { + "both": { + "value": { "fallbackUnimplementedEnabled": true, "fallbackNotFoundEnabled": true } + }, + "one": { "value": { "fallbackNotFoundEnabled": true } } + } + } + } + }, + "responses": { + "200": { "$ref": "#/components/responses/ApiFallbackState" }, + "400": { + "description": "`invalid-request`, with no state change: the body is not a JSON object, is present but empty (`{}`), names a key other than the two toggles, or gives a present key a non-boolean value.", + "content": { + "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + } + }, + "401": { "$ref": "#/components/responses/Unauthenticated" } + } + } + }, + "/actor-runtime/events/{runId}": { + "get": { + "operationId": "connectRunEvents", + "summary": "The run's platform events websocket", + "description": "A websocket upgrade, not a JSON response - the channel the Apify SDKs consume as the platform events socket. It carries `systemInfo` once a second, a one-off `aborting`-plus-`persistState` pair on a graceful abort, and a one-off `migrating` frame when a migration is triggered. Each frame is a single text message, `{\"name\": \"...\", \"data\": {...}}`.\n\nUnauthenticated by decision: the run id in the path is the only thing it scopes on, and a connection only ever receives that run's own frames. An unknown or already-terminal run id gets a completed upgrade followed immediately by a `1008` close with a reason, never a non-101 HTTP status - the Python SDK treats a refused first connection as fatal to the Actor. A connection to a live run stays open until the run ends, when the server closes it with `1000`. A migration or reboot restart is not the run ending: the restarted container reconnects to the same path.\n\nThis operation is served by the HTTP server's upgrade handler rather than by a route, so a plain `GET` that is not a websocket handshake answers `426 upgrade-required`.", + "security": [], + "x-actor-runtime-transport": "websocket", + "parameters": [{ "$ref": "#/components/parameters/RunId" }], + "responses": { + "101": { + "description": "Switching protocols - the websocket is open. Cross-run isolation is structural; an unknown or terminal run is closed with `1008` right after the upgrade completes." + }, + "426": { + "description": "`upgrade-required` - the request reached this path without a websocket handshake.", + "content": { + "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + } + } + } + } + } + }, + "components": { + "securitySchemes": { + "bearerAuth": { + "type": "http", + "scheme": "bearer", + "description": "The same per-token identity every `/v2` route uses: any non-empty token authenticates as that token's own user, created on first sighting." + }, + "tokenQuery": { + "type": "apiKey", + "in": "query", + "name": "token", + "description": "The same token as `bearerAuth`, passed as a query parameter instead." + } + }, + "parameters": { + "ActorId": { + "name": "actorId", + "in": "path", + "required": true, + "description": "The Actor's id, its plain `name`, or `username~name` - the same forms every other endpoint on this API accepts.", + "schema": { "type": "string" } + }, + "RunId": { + "name": "runId", + "in": "path", + "required": true, + "description": "The run's id.", + "schema": { "type": "string" } + } + }, + "responses": { + "Unauthenticated": { + "description": "`user-not-authenticated` - no token was presented. No state change.", + "content": { "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } } + }, + "RecordNotFound": { + "description": "`record-not-found` - no such record, or it belongs to another user. Ownership scoping is not distinguishable from absence, by design.", + "content": { "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } } + }, + "ApiFallbackState": { + "description": "The fallback state, as of immediately after the call.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { "data": { "$ref": "#/components/schemas/ApiFallbackState" } } + } + } + } + } + }, + "schemas": { + "ErrorResponse": { + "type": "object", + "required": ["error"], + "description": "The error envelope every endpoint on this API shares. `type` is what clients branch on.", + "properties": { + "error": { + "type": "object", + "required": ["type", "message"], + "properties": { + "type": { "type": "string" }, + "message": { "type": "string" } + } + } + } + }, + "OpenApiDocument": { + "type": "object", + "description": "An OpenAPI 3.1 document - this one.", + "required": ["openapi", "info", "paths"], + "properties": { + "openapi": { "type": "string" }, + "info": { "type": "object" }, + "paths": { "type": "object" } + }, + "additionalProperties": true + }, + "DevFolderStatus": { + "type": "object", + "required": ["localDevFolder"], + "properties": { + "localDevFolder": { + "type": ["string", "null"], + "description": "The registered absolute host path, or `null` when no folder is registered. The same value the console's Actor detail page shows." + } + } + }, + "DebugModeRequest": { + "type": "object", + "required": ["enabled"], + "additionalProperties": false, + "properties": { + "enabled": { "type": "boolean" }, + "language": { + "type": "string", + "enum": ["auto", "node", "python"], + "default": "auto", + "description": "`auto` classifies the Actor's image at run start." + }, + "port": { + "type": "integer", + "minimum": 1024, + "maximum": 65535, + "description": "Absent means \"use the resolved language's own default port at run start\" (`9229` Node, `5678` Python) - never a stored literal." + } + } + }, + "DebugStatus": { + "type": "object", + "required": ["localDebug"], + "properties": { + "localDebug": { + "oneOf": [ + { + "type": "object", + "required": ["language", "port"], + "properties": { + "language": { "type": "string", "enum": ["auto", "node", "python"] }, + "port": { + "type": "integer", + "description": "For an unresolved `auto`, a nominal default (`5678`) shown purely for display: the port a given run actually publishes depends on that run's own resolved language." + } + } + }, + { "type": "null" } + ], + "description": "`null` when debug mode is off." + } + } + }, + "BrowserViewRequest": { + "type": "object", + "required": ["enabled"], + "additionalProperties": false, + "properties": { + "enabled": { "type": "boolean" }, + "interactive": { + "type": "boolean", + "default": false, + "description": "Also forward mouse and keyboard input to the Actor's browser." + } + } + }, + "BrowserViewStatus": { + "type": "object", + "required": ["localBrowserView"], + "properties": { + "localBrowserView": { + "oneOf": [ + { + "type": "object", + "required": ["interactive"], + "properties": { "interactive": { "type": "boolean" } } + }, + { "type": "null" } + ], + "description": "`null` when browser view is off." + } + } + }, + "ApiFallbackPatch": { + "type": "object", + "minProperties": 1, + "additionalProperties": false, + "properties": { + "fallbackUnimplementedEnabled": { + "type": "boolean", + "description": "Relay a request nothing local answers (a local `404` or `501`)." + }, + "fallbackNotFoundEnabled": { + "type": "boolean", + "description": "Relay a request that reaches a route this runtime serves, but whose record id does not exist locally (`record-not-found`)." + } + } + }, + "ApiFallbackState": { + "type": "object", + "required": ["fallbackUnimplementedEnabled", "fallbackNotFoundEnabled", "upstreamBaseUrl"], + "properties": { + "fallbackUnimplementedEnabled": { "type": "boolean" }, + "fallbackNotFoundEnabled": { "type": "boolean" }, + "upstreamBaseUrl": { + "type": "string", + "format": "uri", + "description": "The platform this runtime would relay to (`https://api.apify.com` by default, or `APIFY_UPSTREAM_API_BASE_URL`). Reported for visibility, read-only: no request body can change it." + } + } + }, + "Run": { + "type": "object", + "description": "An Actor run object, the same shape the emulated platform endpoints (`POST /v2/actor-runs/{runId}/abort`, `.../reboot`) return. `stats` carries `migrationCount`, `rebootCount`, `restartCount` and `resurrectCount`.", + "required": ["id", "status"], + "properties": { + "id": { "type": "string" }, + "actId": { "type": "string" }, + "status": { "type": "string" }, + "stats": { "type": "object" } + }, + "additionalProperties": true + } + } + } +} diff --git a/src/api/routes/actor-runtime-spec.ts b/src/api/routes/actor-runtime-spec.ts new file mode 100644 index 00000000..5bed8359 --- /dev/null +++ b/src/api/routes/actor-runtime-spec.ts @@ -0,0 +1,107 @@ +/** + * The `/actor-runtime/*` namespace's self-description and its terminal handler - the two halves of + * "the OpenAPI document is what the namespace *is*" (`api/actor-runtime-spec.ts`). + * + * `mountActorRuntimeSpec` serves the document itself and is deliberately registered *before* the + * namespace router's `auth()` (`server.ts`): the document is static, identical for every caller and + * carries no user data, so a client can identify a local Actor runtime and enumerate its + * runtime-specific endpoints before it has a token. + * + * `mountActorRuntimeUnmatched` is registered last, after every real route, and answers everything that + * fell through from the document alone. Without it those requests would reach the app-level catch-all + * in `server.ts`, which reads the *platform* spec table and can only ever call them plain `404`s. + */ +import type { Router } from 'express'; + +import { sendData, sendError } from '../envelope.js'; +import { h } from '../handler.js'; +import { + ACTOR_RUNTIME_OPENAPI, + ACTOR_RUNTIME_OPERATIONS, + actorRuntimeOperationsAtPath, + matchActorRuntimeOperation, + routerPathOf, +} from '../actor-runtime-spec.js'; + +/** Both mounts (`/actor-runtime` and `/v2/actor-runtime`) are the same router, so a request's path + * within the namespace is all that identifies it - `req.baseUrl` differs between the two and is + * deliberately not used. */ +function namespacePath(routerRelativePath: string): string { + return `/actor-runtime${routerRelativePath}`; +} + +export function mountActorRuntimeSpec(router: Router): void { + // `{data}`-enveloped, like every other JSON response on this API: `apify api GET /actor-runtime` + // goes through apify-client-js, which unwraps `data` and would otherwise print `undefined`. + router.get( + '/', + h(async (_req, res) => { + sendData(res, ACTOR_RUNTIME_OPENAPI); + }), + ); + + // The same document unenveloped, for OpenAPI tooling pointed straight at the URL - one of the + // documented exceptions to the envelope rule (`api.md`). + router.get( + '/openapi.json', + h(async (_req, res) => { + res.status(200).json(ACTOR_RUNTIME_OPENAPI); + }), + ); + + // Every websocket operation the document declares is served by the HTTP server's `upgrade` event + // (`events-ws.ts`), never by Express - so a request that reaches Express on one of those paths is a + // plain request where a handshake was expected. Registered here, from the document itself, rather + // than in the unmatched handler below, because those operations declare no security: answering a + // missing token with `401` would contradict the document the same request can read. + for (const operation of ACTOR_RUNTIME_OPERATIONS) { + // A websocket handshake is a `GET` by protocol, so that is the only method a websocket operation + // can be declared under; any other method on the same path falls through to the `405` below. + if (operation.transport !== 'websocket' || operation.method !== 'GET') continue; + const path = routerPathOf(operation); + router.get( + path, + h(async (req, res) => { + sendError( + res, + 426, + 'upgrade-required', + `${req.method} ${namespacePath(req.path)} is a websocket endpoint - upgrade required`, + ); + }), + ); + } +} + +export function mountActorRuntimeUnmatched(router: Router): void { + router.use((req, res) => { + const path = namespacePath(req.path); + + if (matchActorRuntimeOperation(req.method, path)) { + // Documented, yet no route matched it: a wiring bug, not a caller error. Answered the same + // way the platform surface answers a spec path with nothing behind it, and covered by a test + // that exercises every documented operation. + sendError(res, 501, 'not-implemented', `${req.method} ${path} is not implemented by this runtime`); + return; + } + + const allowed = actorRuntimeOperationsAtPath(path).map((entry) => entry.method); + if (allowed.length > 0) { + res.setHeader('Allow', allowed.join(', ')); + sendError( + res, + 405, + 'method-not-allowed', + `${req.method} is not allowed on ${path} - allowed: ${allowed.join(', ')}`, + ); + return; + } + + sendError( + res, + 404, + 'not-found', + `${req.method} ${path} is not part of the Actor runtime API - GET /actor-runtime lists every endpoint it has`, + ); + }); +} diff --git a/src/api/server.ts b/src/api/server.ts index e75a734b..be5dac6e 100644 --- a/src/api/server.ts +++ b/src/api/server.ts @@ -18,6 +18,7 @@ import { mountDebugMode } from './routes/debug-mode.js'; import { mountBrowserView } from './routes/browser-view.js'; import { mountMigrate } from './routes/migrate.js'; import { mountApiFallback } from './routes/api-fallback.js'; +import { mountActorRuntimeSpec, mountActorRuntimeUnmatched } from './routes/actor-runtime-spec.js'; import { attemptFallback, type LocalError } from '../services/api-fallback.js'; import type { Driver } from '../driver/types.js'; @@ -48,13 +49,23 @@ export function createApiServer(deps: ApiServerDeps): Express { // module mounted on this router (`mountDevFolder`, `mountMigrate`, `mountApiFallback`) rather than each registering // its own - they are the same router instance, so a second registration would just run `auth()` // twice per request for no benefit. + // What the namespace contains is specified by `openapi/actor-runtime.json`, which the runtime serves + // from itself (`routes/actor-runtime-spec.ts`) and decides its own 404/405/426 from. const actorRuntime = express.Router(); + // Registered before this router's `auth()` on purpose, so the namespace can be enumerated without a + // token - see `routes/actor-runtime-spec.ts`. Everything after `auth()` below is authenticated as + // usual. + mountActorRuntimeSpec(actorRuntime); actorRuntime.use(auth()); mountDevFolder(actorRuntime, deps); mountDebugMode(actorRuntime); mountBrowserView(actorRuntime); mountMigrate(actorRuntime, deps); mountApiFallback(actorRuntime); + // Last on this router: everything that matched no route above is answered from the namespace's own + // OpenAPI document rather than falling through to the app-level catch-all, which knows only the + // emulated platform surface. + mountActorRuntimeUnmatched(actorRuntime); app.use('/actor-runtime', actorRuntime); // Also served at `/v2/actor-runtime/*` - the *same* router instance, no duplicated route logic - solely // because `apify api`'s own URL-building hardcodes a `/v2`-suffixed base (`${baseUrl}/${endpoint}`, diff --git a/test/integration/actor-runtime-spec.test.ts b/test/integration/actor-runtime-spec.test.ts new file mode 100644 index 00000000..ca7882ac --- /dev/null +++ b/test/integration/actor-runtime-spec.test.ts @@ -0,0 +1,101 @@ +/** + * The runtime serves its own `/actor-runtime/*` OpenAPI document, and everything that document does + * not describe is answered from the document too (`requirements/api.md`, "Actor runtime API"). + * + * The load-bearing test here is "every documented operation is actually served": it is what keeps the + * document and the routes from drifting apart, in either direction - a route added without a + * document entry loses its 404/405 contract, a document entry added without a route answers `501`. + */ +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import axios, { type AxiosResponse } from 'axios'; + +import { startTestServer, type TestServerHandle } from './helpers/test-server.js'; +import { ACTOR_RUNTIME_OPERATIONS } from '../../src/api/actor-runtime-spec.js'; + +describe('Actor runtime API self-description', () => { + let server: TestServerHandle; + + beforeEach(async () => { + server = await startTestServer(); + }); + + afterEach(async () => { + await server.close(); + }); + + function call(method: string, path: string, withToken = true): Promise { + return axios.request({ + method, + url: `${server.baseUrl}${path}`, + headers: withToken ? { Authorization: `Bearer ${server.token}` } : {}, + validateStatus: () => true, + }); + } + + it('serves the document at GET /actor-runtime, enveloped, without a token', async () => { + const res = await call('get', '/actor-runtime', false); + expect(res.status).toBe(200); + expect(res.data.data.openapi).toMatch(/^3\.1\./); + expect(Object.keys(res.data.data.paths)).toContain('/actor-runtime/dev-folder/{actorId}'); + }); + + it('serves the bare document at GET /actor-runtime/openapi.json, for OpenAPI tooling', async () => { + const bare = await call('get', '/actor-runtime/openapi.json', false); + const enveloped = await call('get', '/actor-runtime'); + expect(bare.status).toBe(200); + expect(bare.data).not.toHaveProperty('data'); + expect(bare.data).toEqual(enveloped.data.data); + }); + + it('serves the document identically at both mounts', async () => { + const canonical = await call('get', '/actor-runtime'); + const alias = await call('get', '/v2/actor-runtime'); + expect(alias.status).toBe(200); + expect(alias.data).toEqual(canonical.data); + }); + + // One request per documented HTTP operation, with placeholder ids and no body: whatever the + // endpoint makes of that is its own business (`400`, `record-not-found`, ...), as long as the + // request reached a route at all rather than the namespace's unmatched handler. + const httpOperations = ACTOR_RUNTIME_OPERATIONS.filter((operation) => operation.transport === 'http'); + it.each(httpOperations.map((operation) => [`${operation.method} ${operation.path}`, operation] as const))( + '%s is served by a route, not by the unmatched handler', + async (_label, operation) => { + const path = operation.path.replace(/\{[^}]+\}/g, 'does-not-exist-id'); + const res = await call(operation.method.toLowerCase(), path); + expect(res.status).not.toBe(501); + expect(res.status).not.toBe(405); + expect(res.status).not.toBe(426); + if (res.status === 404) expect(res.data.error.type).toBe('record-not-found'); + }, + ); + + it('answers an undocumented path in the namespace with 404 not-found, at both mounts', async () => { + for (const path of ['/actor-runtime/does-not-exist', '/v2/actor-runtime/does-not-exist']) { + const res = await call('get', path); + expect(res.status).toBe(404); + expect(res.data.error.type).toBe('not-found'); + // The message points at the endpoint that lists what the namespace does have. + expect(res.data.error.message).toContain('GET /actor-runtime'); + } + }); + + it('answers an undocumented method on a documented path with 405 and an Allow header', async () => { + const res = await call('get', '/actor-runtime/debug/does-not-exist-id'); + expect(res.status).toBe(405); + expect(res.data.error.type).toBe('method-not-allowed'); + expect(res.headers.allow).toBe('POST'); + + const fallback = await call('put', '/actor-runtime/api-fallback'); + expect(fallback.status).toBe(405); + expect(fallback.headers.allow?.split(', ').sort()).toEqual(['GET', 'POST']); + }); + + it('answers a plain request to the events websocket with 426, with or without a token', async () => { + for (const withToken of [true, false]) { + const res = await call('get', '/actor-runtime/events/some-run', withToken); + expect(res.status).toBe(426); + expect(res.data.error.type).toBe('upgrade-required'); + } + }); +}); diff --git a/test/integration/api-fallback.test.ts b/test/integration/api-fallback.test.ts index d4962bb2..2fcb113d 100644 --- a/test/integration/api-fallback.test.ts +++ b/test/integration/api-fallback.test.ts @@ -503,11 +503,13 @@ describe('api-fallback: eligibility, relay, and fail-closed behaviour', () => { process.env.APIFY_UPSTREAM_API_BASE_URL = stub.baseUrl; try { // Express's own default mount matching is case-insensitive, so this reaches the same - // `actor-runtime` sub-router as the lowercase path above, finds no matching PUT route, and - // falls through to the terminal catch-all with the original casing intact in `originalUrl`. + // `actor-runtime` sub-router as the lowercase path above and finds no matching PUT route. + // The namespace's own unmatched handler answers it from the runtime's OpenAPI document + // (`405`, since the path itself is documented for GET/POST) and never reaches the + // app-level catch-all - either way, nothing about it is eligible for relaying. const res = await call('put', '/v2/ACTOR-RUNTIME/api-fallback'); - expect(res.status).toBe(404); - expect(res.data.error.type).toBe('not-found'); + expect(res.status).toBe(405); + expect(res.data.error.type).toBe('method-not-allowed'); expect(res.headers['x-actor-runtime-fallback']).toBeUndefined(); expect(stub.hitCount()).toBe(0); } finally { diff --git a/test/unit/actor-runtime-spec.test.ts b/test/unit/actor-runtime-spec.test.ts new file mode 100644 index 00000000..61681de0 --- /dev/null +++ b/test/unit/actor-runtime-spec.test.ts @@ -0,0 +1,147 @@ +/** + * The `/actor-runtime/*` namespace's OpenAPI document is the source of truth for what the namespace + * contains (`src/api/actor-runtime-spec.ts`), so these guard the document itself and the lookups the + * server drives from it. That the documented operations are actually *served* is asserted against a + * running server in `test/integration/actor-runtime-spec.test.ts`. + */ +import { readFileSync } from 'node:fs'; +import { describe, expect, it } from 'vitest'; + +import { + ACTOR_RUNTIME_OPENAPI, + ACTOR_RUNTIME_OPERATIONS, + actorRuntimeOperationsAtPath, + matchActorRuntimeOperation, + routerPathOf, +} from '../../src/api/actor-runtime-spec.js'; + +describe('the Actor runtime OpenAPI document', () => { + it('is an OpenAPI 3.1 document describing only the /actor-runtime namespace', () => { + expect(ACTOR_RUNTIME_OPENAPI.openapi).toMatch(/^3\.1\./); + expect(Object.keys(ACTOR_RUNTIME_OPENAPI.paths).length).toBeGreaterThan(0); + for (const path of Object.keys(ACTOR_RUNTIME_OPENAPI.paths)) { + expect(path === '/actor-runtime' || path.startsWith('/actor-runtime/')).toBe(true); + } + }); + + it("carries the runtime's own version, so a client can tell two runtimes apart", () => { + const { version } = JSON.parse(readFileSync(new URL('../../package.json', import.meta.url), 'utf8')) as { + version: string; + }; + expect(ACTOR_RUNTIME_OPENAPI.info.version).toBe(version); + }); + + it('gives every operation a unique operationId and a summary', () => { + const operationIds = ACTOR_RUNTIME_OPERATIONS.map((operation) => operation.operationId); + expect(new Set(operationIds).size).toBe(operationIds.length); + for (const operation of ACTOR_RUNTIME_OPERATIONS) { + expect(operation.operationId).not.toBe(''); + expect(operation.summary).not.toBe(''); + } + }); + + it('describes every endpoint the namespace has', () => { + const described = ACTOR_RUNTIME_OPERATIONS.map((operation) => `${operation.method} ${operation.path}`).sort(); + expect(described).toEqual( + [ + 'GET /actor-runtime', + 'GET /actor-runtime/openapi.json', + 'GET /actor-runtime/api-fallback', + 'POST /actor-runtime/api-fallback', + 'POST /actor-runtime/browser-view/{actorId}', + 'POST /actor-runtime/debug/{actorId}', + 'POST /actor-runtime/dev-folder/{actorId}', + 'POST /actor-runtime/migrate/{runId}', + 'GET /actor-runtime/events/{runId}', + ].sort(), + ); + }); + + it('declares at least one response per operation, and resolves every $ref it uses', () => { + const document = ACTOR_RUNTIME_OPENAPI as unknown as Record; + + for (const [path, pathItem] of Object.entries(ACTOR_RUNTIME_OPENAPI.paths)) { + for (const [method, operation] of Object.entries(pathItem)) { + const responses = (operation as { responses?: Record }).responses ?? {}; + expect(Object.keys(responses).length, `${method} ${path} declares no response`).toBeGreaterThan(0); + } + } + + // A typo'd `$ref` is the one way this document can be structurally broken while still parsing as + // JSON, and nothing else in the suite would notice. + const refsIn = (value: unknown): string[] => { + if (Array.isArray(value)) return value.flatMap(refsIn); + if (typeof value !== 'object' || value === null) return []; + return Object.entries(value).flatMap(([key, child]) => + key === '$ref' && typeof child === 'string' ? [child] : refsIn(child), + ); + }; + const resolve = (ref: string): unknown => + ref + .replace(/^#\//, '') + .split('/') + .reduce((node, segment) => (node as Record | undefined)?.[segment], document); + + const refs = refsIn(document); + expect(refs.length).toBeGreaterThan(0); + for (const ref of refs) { + expect(ref.startsWith('#/'), `${ref} is not a local reference`).toBe(true); + expect(resolve(ref), `${ref} does not resolve`).toBeDefined(); + } + }); + + it('marks the events endpoint as the one websocket transport', () => { + const websocketPaths = ACTOR_RUNTIME_OPERATIONS.filter((operation) => operation.transport === 'websocket').map( + (operation) => operation.path, + ); + expect(websocketPaths).toEqual(['/actor-runtime/events/{runId}']); + }); +}); + +describe('matchActorRuntimeOperation', () => { + it('matches a documented operation through its {param} segments', () => { + expect(matchActorRuntimeOperation('POST', '/actor-runtime/debug/abc123')?.operationId).toBe( + 'setActorDebugMode', + ); + expect(matchActorRuntimeOperation('post', '/actor-runtime/debug/abc123')?.operationId).toBe( + 'setActorDebugMode', + ); + }); + + it('matches the namespace root with or without a trailing slash', () => { + expect(matchActorRuntimeOperation('GET', '/actor-runtime')?.operationId).toBe('getActorRuntimeSpecification'); + expect(matchActorRuntimeOperation('GET', '/actor-runtime/')?.operationId).toBe('getActorRuntimeSpecification'); + }); + + it('is sensitive to the method', () => { + expect(matchActorRuntimeOperation('GET', '/actor-runtime/debug/abc123')).toBeUndefined(); + }); + + it('is sensitive to the segment count, so no path is matched by a prefix of it', () => { + expect(matchActorRuntimeOperation('POST', '/actor-runtime/debug')).toBeUndefined(); + expect(matchActorRuntimeOperation('POST', '/actor-runtime/debug/abc123/extra')).toBeUndefined(); + }); + + it('does not match an undocumented path at all', () => { + expect(matchActorRuntimeOperation('GET', '/actor-runtime/does-not-exist')).toBeUndefined(); + expect(actorRuntimeOperationsAtPath('/actor-runtime/does-not-exist')).toEqual([]); + }); + + it('reports every method documented on a path, which is what the Allow header is built from', () => { + expect( + actorRuntimeOperationsAtPath('/actor-runtime/api-fallback') + .map((entry) => entry.method) + .sort(), + ).toEqual(['GET', 'POST']); + }); +}); + +describe('routerPathOf', () => { + it('turns a documented path template into the Express path the router registers', () => { + const events = matchActorRuntimeOperation('GET', '/actor-runtime/events/some-run')!; + expect(routerPathOf(events)).toBe('/events/:runId'); + + const root = matchActorRuntimeOperation('GET', '/actor-runtime')!; + expect(routerPathOf(root)).toBe('/'); + }); +}); From 9bd30c53022b91bd5df0a27acbc5afdefdaf2a47 Mon Sep 17 00:00:00 2001 From: Josef Prochazka Date: Fri, 11 Sep 2026 11:36:53 +0000 Subject: [PATCH 2/3] docs: Move the runtime API's behaviour into the specification, not just its schemas The first pass moved the request/response shapes into the OpenAPI document but left the behavioural prose in `requirements/api.md`, so the requirements barely shrank - the point of the exercise. All of it now lives in the document, where each operation's own `description` carries it: migration emulation (window, counters, what an abort during the window does), the whole upstream-fallback contract (eligibility, all-methods, marker headers, fail-closed, token forwarding), the dev-folder re-check at run start, and the events channel's lifecycle. Runtime behaviour grafted onto otherwise faithful platform endpoints - `?devFolder=false`, `?gracefully=`, reboot - is listed at the document root under `x-actor-runtime-platform-notes`, so a client enumerating the document still sees every local deviation. `requirements/api.md`'s Actor-runtime section is now 22 lines instead of 174: what the namespace is, where its specification lives, how it is served, and what an undescribed path answers. Code comments and tests that cited the deleted section headings now cite the operation in the document instead. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016VvUV6661cyYbxoHZv2Pby --- requirements/api.md | 166 ++-------- requirements/cli.md | 2 +- requirements/console.md | 2 +- src/api/openapi/actor-runtime.json | 417 +++++++++++++++++++----- src/api/routes/api-fallback.ts | 2 +- src/api/routes/migrate.ts | 2 +- src/console/server.ts | 2 +- src/services/api-fallback.ts | 4 +- src/services/migrations.ts | 2 +- src/services/runs.ts | 2 +- test/integration/api-fallback.test.ts | 10 +- test/integration/graceful-abort.test.ts | 10 +- test/integration/migration.test.ts | 2 +- test/unit/actor-runtime-spec.test.ts | 16 + 14 files changed, 396 insertions(+), 243 deletions(-) diff --git a/requirements/api.md b/requirements/api.md index 6c8cf52e..1a8aa448 100644 --- a/requirements/api.md +++ b/requirements/api.md @@ -131,147 +131,25 @@ # Actor runtime API -- `/actor-runtime/*` is the API controlling functions specific to the local Actor runtime: developer - conveniences (live dev folder, debug mode, browser view, migration emulation, upstream API fallback, - the per-run events channel) that the real Apify platform API has no counterpart for. -- **The namespace has its own OpenAPI specification**, committed at `src/api/openapi/actor-runtime.json`. - That document is the normative contract for every endpoint in it - paths, methods, request bodies, - response payloads, per-rejection error `type`s, and worked examples. This file does not repeat it: - the sections below state only what OpenAPI cannot express (behaviour over time, cross-surface - consistency, and the guarantees the fallback and migration features rest on). -- **The runtime serves that specification from itself**, so a client can enumerate what a given - runtime supports rather than hard-coding a list: - - **`GET /actor-runtime`** - the document in the usual `{data}` envelope, so it reads through - apify-client-js and therefore through `apify api GET /actor-runtime` (`cli.md`). - - **`GET /actor-runtime/openapi.json`** - the same document unenveloped, for OpenAPI tooling - pointed straight at the URL. - - Both are **unauthenticated**, unlike every other endpoint in the namespace: the document is - static, identical for every caller and carries no user data, so a client can identify a local - Actor runtime and enumerate its capabilities before it holds a token. - - The document's `info.version` is the runtime's own version. -- Every endpoint in the namespace is served at both `/actor-runtime/*` (canonical) and - `/v2/actor-runtime/*` (the same routes, reachable a second way purely because `apify api` builds - every URL against a base that already ends in `/v2`). Neither mount is part of the emulated Apify - API, and neither is ever relayed upstream (see "Upstream fallback" below). -- Every endpoint except the two specification endpoints above and the events websocket (below) is - **authenticated** the same way as every `/v2` route and **scoped to the caller's own** Actors/runs, - and none has a **build-first precondition** - a toggle can be set for an Actor that has never been built at all. The endpoints - that set a per-Actor toggle (dev folder, debug mode, browser view) have no separate `GET`: each - response body doubles as the read-back, and each call fully replaces the prior state rather than - merging into it. -- **Anything under `/actor-runtime/*` that the specification does not describe is answered from the - specification**, never from the emulated platform surface: - - an undescribed path answers `404` `not-found`, with a message pointing at `GET /actor-runtime`; - - a described path addressed with an undescribed method answers `405` `method-not-allowed` with an - `Allow` header naming the methods it does have; - - a plain HTTP request to the events websocket path answers `426` `upgrade-required`. -- The console's own dev-folder, debug-mode and browser-view forms (`console.md`) do **not** go through - these endpoints - they post to console-local, unauthenticated routes on the console's own port - but - the two surfaces accept and reject exactly the same inputs with the same outcomes. -- **`POST /v2/actors/:actorId/runs?devFolder=false`** - runs from the built image alone, ignoring the - registered dev folder for that one run only; the registration itself is unchanged. Any other value, - or no parameter, means the default behaviour. A runtime-only query parameter on an otherwise - faithful platform endpoint, so it lives on the platform surface rather than in this namespace; the - specification lists it under `x-actor-runtime-platform-extensions` so a client enumerating the - document still sees it. -- **The events websocket** (`GET /actor-runtime/events/:runId`) carries the run's platform events: - `systemInfo` once a second (`actor-driver.md`), a one-off `aborting`-plus-`persistState` pair under - `?gracefully=` (below), and a one-off `migrating` frame when a migration is triggered ("Migration - emulation" below). It is reachable at exactly this one path on the fixed API port (`system.md`). - - The endpoint has no authentication. The run id in the path is the only thing it scopes on, and a - connection only ever receives that run's own frames; one run never sees another's. - - An unknown or already-terminal run id gets a completed upgrade followed immediately by a `1008` - close with a reason, never a non-101 HTTP status - the Python SDK treats a refused first connection - as fatal to the Actor. - - A connection to a live run stays open until the run ends, when the server closes it with `1000`. It - is never dropped while healthy, except that a graceful runtime shutdown terminates every open - connection along with the rest of the server. A migration/reboot restart is not the run ending: the - restarted container reconnects to the same path. - - The _periodic_ `persistState` is never sent over this channel; both SDKs generate it themselves. - The server sends `persistState` exactly once per graceful abort, alongside `aborting` (matching the - platform), and never alongside `migrating` (the SDKs synthesize that one). - -## Graceful abort (`?gracefully=`) - -- `POST /v2/actor-runs/:runId/abort` accepts an optional `?gracefully=` boolean. -- Omitted or `false`: the run aborts immediately. -- `true` on a running run: the record moves to `ABORTING` at once, an `aborting` frame plus a - `persistState {"isMigrating": false}` frame (in that order, matching the platform) are published on - the run's events channel, and the container is stopped 30 seconds later. The request stays open until - then. -- `true` on a run with no container (still `READY`, or already terminal): behaves as if omitted. -- A second abort arriving during an open window: another `?gracefully=true` joins that window and neither - restarts it nor stops the container early; a non-graceful one escalates and stops the container at once. - -## Migration emulation (`POST /actor-runtime/migrate/:runId`) and reboot - -A platform migration is not a run status: the run stays `RUNNING` while its container is killed and a -new one starts for the same run - same run id, env vars, and default storages, in-memory state gone. -This runtime emulates that observable experience on demand: - -- **`POST /actor-runtime/migrate/:runId`** (also at `/v2/actor-runtime/migrate/:runId`) - authenticated - like the rest of this namespace, scoped to the caller's own runs. The console's run detail view - exposes the same trigger as a Migrate button (`console.md`). - - Publishes a `migrating` frame (empty payload) on the run's events channel immediately, stops the - container 5 seconds later (the platform promises only "a few seconds"), then restarts the same - run. Status stays `RUNNING`; `startedAt`, `finishedAt`, `exitCode`, the default storage ids, and - the container env are unchanged. `stats.migrationCount` increments once per performed stop. - - Responds immediately with the run object (same shape as `abort`/`reboot`). A second call during - the open window joins it: same response, no second frame or window. - - The timeout budget is per run, not per container: a restarted container gets only the remaining - `timeoutSecs`. - - An abort (graceful or hard) landing during the window or restart wins: the run ends `ABORTED`, - never restarted. -- **`POST /v2/actor-runs/:runId/reboot`** - the real platform endpoint the SDKs call from their default - `migrating` handler. Stops and restarts the run's container immediately (no warning frame), cancels an - open migration window, and increments `stats.rebootCount`. A finished run is `403` `job-finished`; a - non-terminal run with no container (`READY`, `ABORTING`) gets the count bump but no restart. -- The run object's `stats` carries `migrationCount`, `rebootCount`, `restartCount`, and `resurrectCount` - (the latter two always `0` here), initialized to `0` at run creation like the platform. -- The run's log is cumulative across restarts, with a one-line marker between the incarnations' output. - -## Upstream fallback (opt-in, off by default, all HTTP methods) - -- Two independent booleans, `fallbackUnimplementedEnabled` and `fallbackNotFoundEnabled`, gate whether - a request this runtime cannot satisfy locally is instead relayed to the real Apify platform. Both - default to `false`, and a restart always brings both back to `false`, regardless of how they were - last set. Either can be on without the other; all four combinations are valid. -- **`GET`/`POST /actor-runtime/api-fallback`** read and change that state; the request and response - shapes are in the specification. `POST` is a partial update - a field the body doesn't mention keeps - its current value - and is the only way to change the state: `upstreamBaseUrl` is reported on every - response for visibility but is read-only (it is the platform this runtime would relay to, - `https://api.apify.com` by default, or the value of `APIFY_UPSTREAM_API_BASE_URL`). A rejected body - changes nothing, not even the fields that would have passed on their own. -- **Which local outcome each toggle covers** (exhaustive - every other error response is never - eligible, under any toggle combination): - - `fallbackUnimplementedEnabled` covers a request the runtime does not serve at all: a local `404` - or `501` response (see "501 vs 404" above). From the caller's point of view both mean "nothing - local answers this", so one toggle covers both. - - `fallbackNotFoundEnabled` covers a request that reaches a route this runtime does serve, but - whose specific record id doesn't exist locally (`record-not-found`, see "Response envelopes" - above). - - Every other error type - `invalid-request`, `user-not-authenticated`, - `cannot-remove-running-run`, `deleting-unfinished-build`, any `dev-folder-*` type, - `internal-error` - is never relayed, regardless of either toggle's state. -- **All HTTP methods are eligible for both toggles, writes included**: a `POST`/`PUT`/`DELETE` that - would otherwise 404/501 locally is relayed exactly like a `GET` when its toggle is on - and, if the - platform accepts it, becomes a real write against the caller's real account. This is a deliberate - consequence of opting in, not an oversight. An eligible request reaches the platform at most once, so - a relayed write is never duplicated. -- **A successful relay** returns the platform's response status and body to the caller unchanged, - marked with two response headers: `x-actor-runtime-fallback: ` naming which platform - served it, and `x-actor-runtime-fallback-trigger: unimplemented` or `record-not-found` naming which - toggle let it through. Only a final `2xx` status counts as successful. -- **Fail-closed guarantee**: anything else - a non-`2xx` response, a timeout, or the platform being - unreachable - reproduces the exact response the caller would have gotten with both toggles off: the - original local error, unchanged, with neither marker header present. The platform's own status or - body is never surfaced to the caller. -- **Only the caller's own presented token is ever forwarded.** A relayed request's `Authorization` - header is always the exact bearer token the caller themselves sent on that request - never a - different or runtime-internal credential, and never sent at all for a request this runtime didn't - authenticate. Enabling either toggle therefore means the caller's own Apify token reaches the - configured `upstreamBaseUrl` on every eligible request; this is the risk being opted into. -- **Never enriches a call that already succeeds locally**: a collection/list endpoint (e.g. - `GET /v2/datasets`) that already returns `200` from local data never consults either toggle and never - gains platform objects. Fallback only ever resolves an otherwise-failing request; it does not make a - local listing "complete". +- `/actor-runtime/*` is the local-runtime-only API: the developer conveniences the Apify platform has no + counterpart for - live dev folder, debug mode, browser view, migration emulation, upstream API + fallback, and the per-run events channel. +- **`src/api/openapi/actor-runtime.json` is the specification for all of it**, and it is normative: + every path, method, request body, response payload, error type and behaviour is stated there and + deliberately not restated here. It also carries, under `x-actor-runtime-platform-notes`, what this + runtime adds to a few otherwise faithful platform endpoints - `?devFolder=false` on run start, + `?gracefully=` on abort, and reboot. +- The runtime serves that document at `GET /actor-runtime` (`{data}`-enveloped, so `apify api` reads it) + and at `GET /actor-runtime/openapi.json` (bare, for OpenAPI tooling). Both are unauthenticated, so a + client can enumerate a runtime before it holds a token (`cli.md`). +- Every endpoint in the namespace is served at both `/actor-runtime/*` and `/v2/actor-runtime/*` - the + same routes, the second mount existing only because `apify api` builds every URL against a base that + already ends in `/v2`. Neither mount is part of the emulated Apify API, and nothing under either is + ever relayed upstream. +- A request under `/actor-runtime/*` that the document does not describe is answered from the document, + never from the emulated platform surface: `404` `not-found` for an undescribed path (the message names + `GET /actor-runtime`), `405` `method-not-allowed` with an `Allow` header for an undescribed method on a + described path, and `426` `upgrade-required` for a plain HTTP request to the events websocket path. +- The console's own dev-folder, debug-mode and browser-view forms (`console.md`) do not go through these + endpoints - they post to console-local, unauthenticated routes on the console's own port - but the two + surfaces accept and reject exactly the same inputs with the same outcomes. diff --git a/requirements/cli.md b/requirements/cli.md index 1937084a..4192dc6c 100644 --- a/requirements/cli.md +++ b/requirements/cli.md @@ -84,7 +84,7 @@ `apify-cli` fetches its actor-templates manifest from the internet. Every later push/call/log-stream/storage-access, and every build of an already-pulled base image, needs no outbound network access (see `system.md`'s offline-after-first-build note) - unless the opt-in - upstream API fallback is enabled (`api.md`, "Upstream fallback"), in which case an eligible local + upstream API fallback is enabled (`api.md`, and the `api-fallback` operation in the runtime API specification it references), in which case an eligible local miss makes one outbound request to the configured upstream instead of failing offline. - The bundled sample Actors crawl the live web (`https://crawlee.dev/` by default), so an `apify call` that runs one of them needs outbound network access from the Actor container even though the diff --git a/requirements/console.md b/requirements/console.md index 412a036c..a7bb0f51 100644 --- a/requirements/console.md +++ b/requirements/console.md @@ -92,7 +92,7 @@ ## Settings page - Every page's header navigation includes a link to `/settings`, the one page for the upstream API - fallback toggles (`api.md`'s "Upstream fallback" section); the link itself shows each toggle's current + fallback toggles (specified by the runtime API specification `api.md` references); the link itself shows each toggle's current value. - `/settings` shows `fallbackUnimplementedEnabled`, `fallbackNotFoundEnabled`, and `upstreamBaseUrl` (the same values the API's toggle endpoint reports), plus a warning that enabling either toggle diff --git a/src/api/openapi/actor-runtime.json b/src/api/openapi/actor-runtime.json index deaa392a..735983cf 100644 --- a/src/api/openapi/actor-runtime.json +++ b/src/api/openapi/actor-runtime.json @@ -4,8 +4,11 @@ "title": "Apify local Actor runtime API", "version": "0.1.0", "summary": "The local-runtime-only endpoints that have no counterpart on the Apify platform.", - "description": "Everything under `/actor-runtime/*` is specific to this local Actor runtime: developer-convenience controls (live dev folder, debug mode, browser view, migration emulation, upstream API fallback) that the real Apify platform API does not have. The emulated subset of the platform API itself is *not* described here - that surface follows https://docs.apify.com/api/openapi.json.\n\nThis document is the normative contract for the namespace; `requirements/api.md` references it rather than repeating it. The runtime serves it from itself, so a client can discover what a given runtime supports: `GET /actor-runtime` returns it in the API's `{data}` envelope (readable through `apify api GET /actor-runtime`), `GET /actor-runtime/openapi.json` returns the bare document for OpenAPI tooling.\n\nA request under `/actor-runtime/*` that this document does not describe never reaches the emulated platform API and is never relayed upstream: an undescribed path answers `404 not-found`, a described path addressed with an undescribed method answers `405 method-not-allowed` with an `Allow` header.", - "license": { "name": "Apache-2.0", "identifier": "Apache-2.0" } + "description": "Everything under `/actor-runtime/*` is specific to this local Actor runtime: developer-convenience controls (live dev folder, debug mode, browser view, migration emulation, upstream API fallback, the per-run events channel) that the real Apify platform API does not have. The emulated subset of the platform API itself is *not* described here - that surface follows https://docs.apify.com/api/openapi.json.\n\nThis document is the normative specification for the namespace: `requirements/api.md` states only that it exists and references it, rather than repeating any of it. What this runtime adds to a handful of otherwise faithful *platform* endpoints is listed under `x-actor-runtime-platform-notes` at the root of this document, so a client enumerating it sees every local deviation in one place.\n\nThe runtime serves this document from itself, so a client can discover what a given runtime supports: `GET /actor-runtime` returns it in the API's `{data}` envelope (readable through `apify api GET /actor-runtime`), `GET /actor-runtime/openapi.json` returns it bare for OpenAPI tooling. Both are unauthenticated; every other operation here is authenticated like every `/v2` route and scoped to the caller's own Actors/runs, except the events websocket, which scopes on its run id alone. Every operation is served at both `/actor-runtime/*` and `/v2/actor-runtime/*` (see `servers`).\n\nA request under `/actor-runtime/*` that this document does not describe never reaches the emulated platform API and is never relayed upstream: an undescribed path answers `404 not-found`, a described path addressed with an undescribed method answers `405 method-not-allowed` with an `Allow` header, and a plain HTTP request to a websocket path answers `426 upgrade-required`.", + "license": { + "name": "Apache-2.0", + "identifier": "Apache-2.0" + } }, "servers": [ { @@ -17,7 +20,14 @@ "description": "Alias mount serving the exact same routes, so `apify api /actor-runtime/...` reaches them: the CLI builds every URL against a base that already ends in `/v2`." } ], - "security": [{ "bearerAuth": [] }, { "tokenQuery": [] }], + "security": [ + { + "bearerAuth": [] + }, + { + "tokenQuery": [] + } + ], "x-actor-runtime-platform-extensions": [ { "method": "POST", @@ -41,7 +51,11 @@ "schema": { "type": "object", "required": ["data"], - "properties": { "data": { "$ref": "#/components/schemas/OpenApiDocument" } } + "properties": { + "data": { + "$ref": "#/components/schemas/OpenApiDocument" + } + } } } } @@ -59,7 +73,11 @@ "200": { "description": "The runtime-specific OpenAPI document, unenveloped.", "content": { - "application/json": { "schema": { "$ref": "#/components/schemas/OpenApiDocument" } } + "application/json": { + "schema": { + "$ref": "#/components/schemas/OpenApiDocument" + } + } } } } @@ -69,17 +87,29 @@ "post": { "operationId": "setActorDevFolder", "summary": "Register or clear an Actor's live dev folder", - "description": "Registers the host directory bind-mounted over the image's working directory on every subsequent run of this Actor, so source edits take effect without a rebuild (`requirements/actor-driver.md`). Submitting `\"\"` clears the registration.\n\nScoped to the caller's own Actors. There is no build-first precondition: registration works for an Actor that has never been built. The response body doubles as the read-back - there is deliberately no separate `GET`.\n\nThe console's own dev-folder form is a console-local, unauthenticated route on the console's port and does not go through this endpoint, but both surfaces accept and reject exactly the same inputs with the same outcomes.", - "parameters": [{ "$ref": "#/components/parameters/ActorId" }], + "description": "Registers the host directory bind-mounted over the image's working directory on every subsequent run of this Actor, so source edits take effect without a rebuild (`requirements/actor-driver.md`). Submitting `\"\"` clears the registration. The path is checked when it is registered and again at every run start, so a folder deleted after registration fails the run rather than running against an empty directory; a single run can opt out with `?devFolder=false` (see `x-actor-runtime-platform-notes`).\n\nScoped to the caller's own Actors. There is no build-first precondition: registration works for an Actor that has never been built. The response body doubles as the read-back - there is deliberately no separate `GET`.\n\nThe console's own dev-folder form is a console-local, unauthenticated route on the console's port and does not go through this endpoint, but both surfaces accept and reject exactly the same inputs with the same outcomes.", + "parameters": [ + { + "$ref": "#/components/parameters/ActorId" + } + ], "requestBody": { "required": true, "description": "A JSON string: the absolute path to register, or `\"\"` to clear.", "content": { "application/json": { - "schema": { "type": "string" }, + "schema": { + "type": "string" + }, "examples": { - "register": { "summary": "Register a folder", "value": "/abs/path/to/src" }, - "clear": { "summary": "Clear the registration", "value": "" } + "register": { + "summary": "Register a folder", + "value": "/abs/path/to/src" + }, + "clear": { + "summary": "Clear the registration", + "value": "" + } } } } @@ -92,7 +122,11 @@ "schema": { "type": "object", "required": ["data"], - "properties": { "data": { "$ref": "#/components/schemas/DevFolderStatus" } } + "properties": { + "data": { + "$ref": "#/components/schemas/DevFolderStatus" + } + } } } } @@ -100,21 +134,37 @@ "400": { "description": "Rejected, with no state change. Error `type` names the reason: `invalid-request` (the body is not a JSON string, or the string is not a valid absolute path), `dev-folder-path-not-found` (the path does not exist on the host), `dev-folder-not-a-directory` (the path exists but is not a directory), `dev-folder-check-failed` (the path could not be verified, for any other reason).", "content": { - "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } } }, - "401": { "$ref": "#/components/responses/Unauthenticated" }, - "404": { "$ref": "#/components/responses/RecordNotFound" }, + "401": { + "$ref": "#/components/responses/Unauthenticated" + }, + "404": { + "$ref": "#/components/responses/RecordNotFound" + }, "500": { "description": "`internal-error` - an operational fault unrelated to the submitted path.", "content": { - "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } } }, "503": { "description": "`dev-folder-check-unavailable` - Docker itself is unreachable, so the path could not be checked.", "content": { - "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } } } } @@ -125,22 +175,39 @@ "operationId": "setActorDebugMode", "summary": "Set or clear an Actor's debug-mode toggle", "description": "While debug mode is on, every run of this Actor starts paused waiting for a debugger, with the attach address printed in the run's own log and the debug port published on `127.0.0.1` (`requirements/actor-driver.md`).\n\nScoped to the caller's own Actors; no build-first precondition. Every accepted call fully replaces the prior state - never a partial merge, so a field the body omits resets to its own default rather than keeping what a previous call set. `{\"enabled\": false}` clears the whole toggle, whatever else the body names. The response doubles as the read-back; there is no separate `GET`.\n\nThe console's own debug-mode form is a console-local, unauthenticated route and does not go through this endpoint, but both surfaces accept and reject exactly the same inputs with the same outcomes.", - "parameters": [{ "$ref": "#/components/parameters/ActorId" }], + "parameters": [ + { + "$ref": "#/components/parameters/ActorId" + } + ], "requestBody": { "required": true, "content": { "application/json": { - "schema": { "$ref": "#/components/schemas/DebugModeRequest" }, + "schema": { + "$ref": "#/components/schemas/DebugModeRequest" + }, "examples": { "enableAutoDetected": { "summary": "Turn on, auto-detect the language", - "value": { "enabled": true } + "value": { + "enabled": true + } }, "enableNode": { "summary": "Turn on for Node on an explicit port", - "value": { "enabled": true, "language": "node", "port": 9229 } + "value": { + "enabled": true, + "language": "node", + "port": 9229 + } }, - "disable": { "summary": "Turn off", "value": { "enabled": false } } + "disable": { + "summary": "Turn off", + "value": { + "enabled": false + } + } } } } @@ -153,16 +220,40 @@ "schema": { "type": "object", "required": ["data"], - "properties": { "data": { "$ref": "#/components/schemas/DebugStatus" } } + "properties": { + "data": { + "$ref": "#/components/schemas/DebugStatus" + } + } }, "examples": { "enabledAutoDetected": { - "value": { "data": { "localDebug": { "language": "auto", "port": 5678 } } } + "value": { + "data": { + "localDebug": { + "language": "auto", + "port": 5678 + } + } + } }, "enabledNode": { - "value": { "data": { "localDebug": { "language": "node", "port": 9229 } } } + "value": { + "data": { + "localDebug": { + "language": "node", + "port": 9229 + } + } + } }, - "disabled": { "value": { "data": { "localDebug": null } } } + "disabled": { + "value": { + "data": { + "localDebug": null + } + } + } } } } @@ -170,11 +261,19 @@ "400": { "description": "`invalid-request`, with no state change: the body is not a JSON object, names an unknown field, misses `enabled` or gives it a non-boolean, sets an invalid `language`, or sets a `port` outside `1024..65535`. Example message for an unknown field: `Unknown field \"prot\" - allowed fields are \"enabled\", \"language\", \"port\".`", "content": { - "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } } }, - "401": { "$ref": "#/components/responses/Unauthenticated" }, - "404": { "$ref": "#/components/responses/RecordNotFound" } + "401": { + "$ref": "#/components/responses/Unauthenticated" + }, + "404": { + "$ref": "#/components/responses/RecordNotFound" + } } } }, @@ -183,19 +282,38 @@ "operationId": "setActorBrowserView", "summary": "Set or clear an Actor's browser-view toggle", "description": "While browser view is on, every run of this Actor prints a viewer URL in its log - a live view of the display the Actor's browser draws on, served on the console's port (`requirements/actor-driver.md`). `interactive` also forwards mouse and keyboard input.\n\nScoped to the caller's own Actors; no build-first precondition. A call fully replaces the prior state; `{\"enabled\": false}` clears it. The response doubles as the read-back; there is no separate `GET`.", - "parameters": [{ "$ref": "#/components/parameters/ActorId" }], + "parameters": [ + { + "$ref": "#/components/parameters/ActorId" + } + ], "requestBody": { "required": true, "content": { "application/json": { - "schema": { "$ref": "#/components/schemas/BrowserViewRequest" }, + "schema": { + "$ref": "#/components/schemas/BrowserViewRequest" + }, "examples": { - "watch": { "summary": "Watch only", "value": { "enabled": true } }, + "watch": { + "summary": "Watch only", + "value": { + "enabled": true + } + }, "interact": { "summary": "Watch and control", - "value": { "enabled": true, "interactive": true } + "value": { + "enabled": true, + "interactive": true + } }, - "disable": { "summary": "Turn off", "value": { "enabled": false } } + "disable": { + "summary": "Turn off", + "value": { + "enabled": false + } + } } } } @@ -208,7 +326,11 @@ "schema": { "type": "object", "required": ["data"], - "properties": { "data": { "$ref": "#/components/schemas/BrowserViewStatus" } } + "properties": { + "data": { + "$ref": "#/components/schemas/BrowserViewStatus" + } + } } } } @@ -216,11 +338,19 @@ "400": { "description": "`invalid-request`, with no state change - any body other than `{\"enabled\": boolean, \"interactive\"?: boolean}`.", "content": { - "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } } }, - "401": { "$ref": "#/components/responses/Unauthenticated" }, - "404": { "$ref": "#/components/responses/RecordNotFound" } + "401": { + "$ref": "#/components/responses/Unauthenticated" + }, + "404": { + "$ref": "#/components/responses/RecordNotFound" + } } } }, @@ -228,8 +358,12 @@ "post": { "operationId": "migrateRun", "summary": "Emulate a platform migration of a running run", - "description": "Gives the run the platform's migration experience: a `migrating` frame on its events channel immediately, its container stopped a few seconds later, then a fresh container for the same run - same run id, env vars and default storages, in-memory state gone. The run never leaves `RUNNING`, and `stats.migrationCount` increments once per performed stop.\n\nScoped to the caller's own runs. Responds immediately with the run object, the same shape `abort`/`reboot` return; a second call during the open window joins it - same response, no second frame or window. The console's run detail view exposes the same trigger as a Migrate button.", - "parameters": [{ "$ref": "#/components/parameters/RunId" }], + "description": "A platform migration is not a run status: the run stays `RUNNING` while its container is killed and a new one starts for the same run - same run id, env vars and default storages, in-memory state gone. This endpoint emulates that observable experience on demand.\n\nIt publishes a `migrating` frame (empty payload) on the run's events channel immediately, stops the container 5 seconds later (the platform promises only \"a few seconds\"), then restarts the same run. Status stays `RUNNING`; `startedAt`, `finishedAt`, `exitCode`, the default storage ids and the container env are unchanged, and `stats.migrationCount` increments once per performed stop. The timeout budget is per run, not per container: a restarted container gets only the remaining `timeoutSecs`. The run's log is cumulative across restarts, with a one-line marker between the incarnations' output.\n\nResponds immediately with the run object, the same shape `abort`/`reboot` return. A second call during the open window joins it: same response, no second frame and no second window. An abort (graceful or hard) landing during the window or the restart wins - the run ends `ABORTED`, never restarted.\n\nThe console's run detail view exposes the same trigger as a Migrate button. The platform's own `POST /v2/actor-runs/{runId}/reboot` is implemented too - see `x-actor-runtime-platform-notes`.", + "parameters": [ + { + "$ref": "#/components/parameters/RunId" + } + ], "responses": { "200": { "description": "The run object, read back after the migration was scheduled.", @@ -238,7 +372,11 @@ "schema": { "type": "object", "required": ["data"], - "properties": { "data": { "$ref": "#/components/schemas/Run" } } + "properties": { + "data": { + "$ref": "#/components/schemas/Run" + } + } } } } @@ -246,17 +384,29 @@ "400": { "description": "`invalid-request` - the run is non-terminal but has no container to migrate (`READY` or `ABORTING`).", "content": { - "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } } }, - "401": { "$ref": "#/components/responses/Unauthenticated" }, + "401": { + "$ref": "#/components/responses/Unauthenticated" + }, "403": { "description": "`job-finished` - the run has already finished.", "content": { - "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } } }, - "404": { "$ref": "#/components/responses/RecordNotFound" } + "404": { + "$ref": "#/components/responses/RecordNotFound" + } } } }, @@ -265,38 +415,60 @@ "operationId": "getApiFallbackState", "summary": "Read the upstream API fallback state", "responses": { - "200": { "$ref": "#/components/responses/ApiFallbackState" }, - "401": { "$ref": "#/components/responses/Unauthenticated" } - } + "200": { + "$ref": "#/components/responses/ApiFallbackState" + }, + "401": { + "$ref": "#/components/responses/Unauthenticated" + } + }, + "description": "Reads both toggles and the platform they would relay to. The toggles' full contract is on the `POST` below." }, "post": { "operationId": "setApiFallbackState", "summary": "Turn upstream API fallback on or off", - "description": "Two independent toggles gate whether a request this runtime cannot satisfy locally is relayed to the real Apify platform instead of failing: `fallbackUnimplementedEnabled` covers a local `404`/`501` (nothing local answers this), `fallbackNotFoundEnabled` covers a `record-not-found` on a route this runtime does serve. Both default to `false` and a restart always brings them back to `false`.\n\nA field the body does not mention keeps its current value. All HTTP methods are eligible for relaying, writes included, and a relayed request carries the caller's own token to the upstream platform - see `requirements/api.md`'s \"Upstream fallback\" section for the full eligibility, fail-closed and token-forwarding contract.", + "description": "Two independent toggles gate whether a request this runtime cannot satisfy locally is relayed to the real Apify platform instead of failing. Both default to `false`, and a restart always brings both back to `false`, regardless of how they were last set. Either can be on without the other; all four combinations are valid. This `POST` is a partial update: a field the body doesn't mention keeps its current value. `upstreamBaseUrl` is reported on every response for visibility but is read-only - no request body can change it.\n\n**Which local outcome each toggle covers** (exhaustive - every other response is never eligible, under any toggle combination):\n- `fallbackUnimplementedEnabled` covers a request the runtime does not serve at all: a local `404` or `501`. From the caller's point of view both mean \"nothing local answers this\", so one toggle covers both.\n- `fallbackNotFoundEnabled` covers a request that reaches a route this runtime does serve, but whose specific record id doesn't exist locally (`record-not-found`).\n- Every other error type - `invalid-request`, `user-not-authenticated`, `cannot-remove-running-run`, `deleting-unfinished-build`, any `dev-folder-*` type, `internal-error` - is never relayed, whatever either toggle is set to. Neither is anything under `/actor-runtime/*` itself, which has no counterpart upstream.\n\n**All HTTP methods are eligible for both toggles, writes included**: a `POST`/`PUT`/`DELETE` that would otherwise 404/501 locally is relayed exactly like a `GET` when its toggle is on - and, if the platform accepts it, becomes a real write against the caller's real account. This is a deliberate consequence of opting in, not an oversight. An eligible request reaches the platform at most once, so a relayed write is never duplicated.\n\n**A successful relay** returns the platform's response status and body to the caller unchanged, marked with two response headers: `x-actor-runtime-fallback: ` naming which platform served it, and `x-actor-runtime-fallback-trigger: unimplemented` or `record-not-found` naming which toggle let it through. Only a final `2xx` status counts as successful.\n\n**Fail-closed guarantee**: anything else - a non-`2xx` response, a timeout, or the platform being unreachable - reproduces the exact response the caller would have gotten with both toggles off: the original local error, unchanged, with neither marker header present. The platform's own status or body is never surfaced to the caller.\n\n**Only the caller's own presented token is ever forwarded.** A relayed request's `Authorization` header is always the exact bearer token the caller themselves sent on that request - never a different or runtime-internal credential, and never sent at all for a request this runtime didn't authenticate. Enabling either toggle therefore means the caller's own Apify token reaches the configured `upstreamBaseUrl` on every eligible request; this is the risk being opted into.\n\n**Never enriches a call that already succeeds locally**: a collection endpoint (e.g. `GET /v2/datasets`) that already returns `200` from local data never consults either toggle and never gains platform objects. Fallback only ever resolves an otherwise-failing request; it does not make a local listing \"complete\".", "requestBody": { "required": true, "description": "A JSON object naming either toggle, or both.", "content": { "application/json": { - "schema": { "$ref": "#/components/schemas/ApiFallbackPatch" }, + "schema": { + "$ref": "#/components/schemas/ApiFallbackPatch" + }, "examples": { "both": { - "value": { "fallbackUnimplementedEnabled": true, "fallbackNotFoundEnabled": true } + "value": { + "fallbackUnimplementedEnabled": true, + "fallbackNotFoundEnabled": true + } }, - "one": { "value": { "fallbackNotFoundEnabled": true } } + "one": { + "value": { + "fallbackNotFoundEnabled": true + } + } } } } }, "responses": { - "200": { "$ref": "#/components/responses/ApiFallbackState" }, + "200": { + "$ref": "#/components/responses/ApiFallbackState" + }, "400": { "description": "`invalid-request`, with no state change: the body is not a JSON object, is present but empty (`{}`), names a key other than the two toggles, or gives a present key a non-boolean value.", "content": { - "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } } }, - "401": { "$ref": "#/components/responses/Unauthenticated" } + "401": { + "$ref": "#/components/responses/Unauthenticated" + } } } }, @@ -307,7 +479,11 @@ "description": "A websocket upgrade, not a JSON response - the channel the Apify SDKs consume as the platform events socket. It carries `systemInfo` once a second, a one-off `aborting`-plus-`persistState` pair on a graceful abort, and a one-off `migrating` frame when a migration is triggered. Each frame is a single text message, `{\"name\": \"...\", \"data\": {...}}`.\n\nUnauthenticated by decision: the run id in the path is the only thing it scopes on, and a connection only ever receives that run's own frames. An unknown or already-terminal run id gets a completed upgrade followed immediately by a `1008` close with a reason, never a non-101 HTTP status - the Python SDK treats a refused first connection as fatal to the Actor. A connection to a live run stays open until the run ends, when the server closes it with `1000`. A migration or reboot restart is not the run ending: the restarted container reconnects to the same path.\n\nThis operation is served by the HTTP server's upgrade handler rather than by a route, so a plain `GET` that is not a websocket handshake answers `426 upgrade-required`.", "security": [], "x-actor-runtime-transport": "websocket", - "parameters": [{ "$ref": "#/components/parameters/RunId" }], + "parameters": [ + { + "$ref": "#/components/parameters/RunId" + } + ], "responses": { "101": { "description": "Switching protocols - the websocket is open. Cross-run isolation is structural; an unknown or terminal run is closed with `1008` right after the upgrade completes." @@ -315,7 +491,11 @@ "426": { "description": "`upgrade-required` - the request reached this path without a websocket handshake.", "content": { - "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } } } } @@ -342,24 +522,40 @@ "in": "path", "required": true, "description": "The Actor's id, its plain `name`, or `username~name` - the same forms every other endpoint on this API accepts.", - "schema": { "type": "string" } + "schema": { + "type": "string" + } }, "RunId": { "name": "runId", "in": "path", "required": true, "description": "The run's id.", - "schema": { "type": "string" } + "schema": { + "type": "string" + } } }, "responses": { "Unauthenticated": { "description": "`user-not-authenticated` - no token was presented. No state change.", - "content": { "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } } + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } }, "RecordNotFound": { "description": "`record-not-found` - no such record, or it belongs to another user. Ownership scoping is not distinguishable from absence, by design.", - "content": { "application/json": { "schema": { "$ref": "#/components/schemas/ErrorResponse" } } } + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } }, "ApiFallbackState": { "description": "The fallback state, as of immediately after the call.", @@ -368,7 +564,11 @@ "schema": { "type": "object", "required": ["data"], - "properties": { "data": { "$ref": "#/components/schemas/ApiFallbackState" } } + "properties": { + "data": { + "$ref": "#/components/schemas/ApiFallbackState" + } + } } } } @@ -384,8 +584,12 @@ "type": "object", "required": ["type", "message"], "properties": { - "type": { "type": "string" }, - "message": { "type": "string" } + "type": { + "type": "string" + }, + "message": { + "type": "string" + } } } } @@ -395,9 +599,15 @@ "description": "An OpenAPI 3.1 document - this one.", "required": ["openapi", "info", "paths"], "properties": { - "openapi": { "type": "string" }, - "info": { "type": "object" }, - "paths": { "type": "object" } + "openapi": { + "type": "string" + }, + "info": { + "type": "object" + }, + "paths": { + "type": "object" + } }, "additionalProperties": true }, @@ -416,7 +626,9 @@ "required": ["enabled"], "additionalProperties": false, "properties": { - "enabled": { "type": "boolean" }, + "enabled": { + "type": "boolean" + }, "language": { "type": "string", "enum": ["auto", "node", "python"], @@ -441,14 +653,19 @@ "type": "object", "required": ["language", "port"], "properties": { - "language": { "type": "string", "enum": ["auto", "node", "python"] }, + "language": { + "type": "string", + "enum": ["auto", "node", "python"] + }, "port": { "type": "integer", "description": "For an unresolved `auto`, a nominal default (`5678`) shown purely for display: the port a given run actually publishes depends on that run's own resolved language." } } }, - { "type": "null" } + { + "type": "null" + } ], "description": "`null` when debug mode is off." } @@ -459,7 +676,9 @@ "required": ["enabled"], "additionalProperties": false, "properties": { - "enabled": { "type": "boolean" }, + "enabled": { + "type": "boolean" + }, "interactive": { "type": "boolean", "default": false, @@ -476,9 +695,15 @@ { "type": "object", "required": ["interactive"], - "properties": { "interactive": { "type": "boolean" } } + "properties": { + "interactive": { + "type": "boolean" + } + } }, - { "type": "null" } + { + "type": "null" + } ], "description": "`null` when browser view is off." } @@ -503,8 +728,12 @@ "type": "object", "required": ["fallbackUnimplementedEnabled", "fallbackNotFoundEnabled", "upstreamBaseUrl"], "properties": { - "fallbackUnimplementedEnabled": { "type": "boolean" }, - "fallbackNotFoundEnabled": { "type": "boolean" }, + "fallbackUnimplementedEnabled": { + "type": "boolean" + }, + "fallbackNotFoundEnabled": { + "type": "boolean" + }, "upstreamBaseUrl": { "type": "string", "format": "uri", @@ -517,13 +746,43 @@ "description": "An Actor run object, the same shape the emulated platform endpoints (`POST /v2/actor-runs/{runId}/abort`, `.../reboot`) return. `stats` carries `migrationCount`, `rebootCount`, `restartCount` and `resurrectCount`.", "required": ["id", "status"], "properties": { - "id": { "type": "string" }, - "actId": { "type": "string" }, - "status": { "type": "string" }, - "stats": { "type": "object" } + "id": { + "type": "string" + }, + "actId": { + "type": "string" + }, + "status": { + "type": "string" + }, + "stats": { + "type": "object" + } }, "additionalProperties": true } } - } + }, + "x-actor-runtime-platform-notes": [ + { + "method": "POST", + "path": "/v2/actors/{actorId}/runs", + "parameter": "devFolder", + "summary": "Run once from the built image alone", + "description": "`?devFolder=false` runs from the built image alone, ignoring the Actor's registered live dev folder for that one run; the registration itself is unchanged. Any other value, or no parameter, means the default behaviour. The run's own log records that it was used." + }, + { + "method": "POST", + "path": "/v2/actor-runs/{runId}/abort", + "parameter": "gracefully", + "summary": "Graceful abort, with the platform’s warning frames and grace period", + "description": "`POST /v2/actor-runs/{runId}/abort` accepts an optional `?gracefully=` boolean. Omitted or `false`: the run aborts immediately. `true` on a running run: the record moves to `ABORTING` at once, an `aborting` frame plus a `persistState {\"isMigrating\": false}` frame (in that order, matching the platform) are published on the run's events channel, and the container is stopped 30 seconds later - the request stays open until then. `true` on a run with no container (still `READY`, or already terminal) behaves as if omitted. A second abort arriving during an open window: another `?gracefully=true` joins that window and neither restarts it nor stops the container early; a non-graceful one escalates and stops the container at once." + }, + { + "method": "POST", + "path": "/v2/actor-runs/{runId}/reboot", + "summary": "Reboot, the endpoint the SDKs call from their own migrating handler", + "description": "The real platform endpoint the SDKs call from their default `migrating` handler, implemented here: it stops and restarts the run's container immediately (no warning frame), cancels an open migration window, and increments `stats.rebootCount`. A finished run is `403 job-finished`; a non-terminal run with no container (`READY`, `ABORTING`) gets the count bump but no restart. The run object's `stats` carries `migrationCount`, `rebootCount`, `restartCount` and `resurrectCount` (the latter two always `0` here), initialized to `0` at run creation like the platform." + } + ] } diff --git a/src/api/routes/api-fallback.ts b/src/api/routes/api-fallback.ts index 2acf6f64..ee7381ab 100644 --- a/src/api/routes/api-fallback.ts +++ b/src/api/routes/api-fallback.ts @@ -1,5 +1,5 @@ /** - * `GET`/`POST /actor-runtime/api-fallback` (`api.md`'s "Upstream fallback" section) - mounted on the + * `GET`/`POST /actor-runtime/api-fallback` (specified by the `getApiFallbackState`/`setApiFallbackState` operations in `../openapi/actor-runtime.json`) - mounted on the * same `/actor-runtime` sub-router `dev-folder.ts` already registers on, so it shares that router's * single `auth()` registration (`server.ts`) rather than adding its own, and is served at both mounts * (`/actor-runtime/api-fallback` and `/v2/actor-runtime/api-fallback`) the same way the dev-folder route diff --git a/src/api/routes/migrate.ts b/src/api/routes/migrate.ts index 000f6c67..2ff78bfb 100644 --- a/src/api/routes/migrate.ts +++ b/src/api/routes/migrate.ts @@ -1,6 +1,6 @@ /** * `POST /actor-runtime/migrate/:runId` - triggers an emulated migration of one run - * (`requirements/api.md`, "Migration emulation"). In the local-runtime-only namespace because the real + * (the `migrateRun` operation in `src/api/openapi/actor-runtime.json`). In the local-runtime-only namespace because the real * platform has no migrate API. */ import type { Router } from 'express'; diff --git a/src/console/server.ts b/src/console/server.ts index ce646bd7..34a4c689 100644 --- a/src/console/server.ts +++ b/src/console/server.ts @@ -13,7 +13,7 @@ * object's owner `userId` (`console.md`: "Frontend shows for each object the owner (userId)"). The * dev-folder form, the debug-mode form, and the Migrate button all write cross-user the same way - a * deliberate deviation from the API's own strictly-owner-scoped writes, not an accident; the `/settings` - * form is runtime-global by nature (`api.md`'s "Upstream fallback" section), so ownership doesn't apply + * form is runtime-global by nature (`src/api/openapi/actor-runtime.json`'s `api-fallback` operations), so ownership doesn't apply * to it at all. */ import { createRequire } from 'node:module'; diff --git a/src/services/api-fallback.ts b/src/services/api-fallback.ts index fb8020e4..b758948e 100644 --- a/src/services/api-fallback.ts +++ b/src/services/api-fallback.ts @@ -1,5 +1,5 @@ /** - * Upstream API fallback (`api.md`'s "Upstream fallback" section): when a call locally misses - either + * Upstream API fallback (the `setApiFallbackState` operation in `src/api/openapi/actor-runtime.json`): when a call locally misses - either * because nothing in this runtime serves the path/method at all, or because it does but the specific * record id doesn't exist - and the matching toggle is on, the request is replayed against the real * Apify platform instead of failing, and a successful reply's status/body/headers are relayed back @@ -84,7 +84,7 @@ const EXCLUDED_RESPONSE_HEADERS = new Set([ export type FallbackTrigger = 'unimplemented' | 'record-not-found'; -/** The exhaustive mapping from a local error's `type` to the toggle that gates it (`api.md`). Every +/** The exhaustive mapping from a local error's `type` to the toggle that gates it (`openapi/actor-runtime.json`). Every * other error `type` - `invalid-request`, `user-not-authenticated`, `cannot-remove-running-run`, * `deleting-unfinished-build`, any `dev-folder-*` type, `internal-error` - is never eligible, `null`. */ function triggerForErrorType(type: string): FallbackTrigger | null { diff --git a/src/services/migrations.ts b/src/services/migrations.ts index 0bdbadbc..7771efe4 100644 --- a/src/services/migrations.ts +++ b/src/services/migrations.ts @@ -1,5 +1,5 @@ /** - * Emulated migrations and reboots (`requirements/api.md`, "Migration emulation"). A migration is not a + * Emulated migrations and reboots (the `migrateRun` operation in `src/api/openapi/actor-runtime.json`). A migration is not a * run status - the run keeps running while its container is replaced, so this module only tracks which * container stops must restart the run instead of finishing it. */ diff --git a/src/services/runs.ts b/src/services/runs.ts index 585c52aa..79cf79b1 100644 --- a/src/services/runs.ts +++ b/src/services/runs.ts @@ -434,7 +434,7 @@ function remainingTimeoutSecs(record: RunRecord): number { * * `gracefully` on a `RUNNING` run publishes the platform's `aborting` + `persistState` frame pair and * waits `GRACEFUL_ABORT_WINDOW_MS` before stopping; other states take the immediate path. A second - * concurrent graceful abort joins the window rather than restarting it - see `requirements/api.md`. + * concurrent graceful abort joins the window rather than restarting it - see the abort entry in `src/api/openapi/actor-runtime.json`'s `x-actor-runtime-platform-notes`. * * Both flags come from `onBeforeTransition`, read inside the same mutex-serialized write that performs * the transition: a preceding `get()` could observe a stale status, and only the hook can tell "this call diff --git a/test/integration/api-fallback.test.ts b/test/integration/api-fallback.test.ts index 2fcb113d..a3861090 100644 --- a/test/integration/api-fallback.test.ts +++ b/test/integration/api-fallback.test.ts @@ -1,5 +1,5 @@ /** - * Covers the upstream API fallback (`api.md`'s "Upstream fallback" section, `services/api-fallback.ts`): + * Covers the upstream API fallback (the `api-fallback` operations in `src/api/openapi/actor-runtime.json`, `services/api-fallback.ts`): * the `GET`/`POST /actor-runtime/api-fallback` toggle-state endpoint, the eligibility mapping (both * toggles, in isolation and together), the fail-closed guarantee, own-token-only forwarding, and the two * marker headers - against a stubbed upstream, exactly the pattern @@ -138,7 +138,7 @@ async function warmUpIdentity(baseUrl: string, token: string): Promise { } /** A fixed `2xx` JSON response with a distinguishing header - the shape every successful-relay test - * below checks the caller receives unchanged (`api.md`'s "A successful relay" bullet). */ + * below checks the caller receives unchanged (the specification's "A successful relay" paragraph). */ function fixedOkResponse(distinguishingValue: string) { return () => ({ status: 200, @@ -380,7 +380,7 @@ describe('api-fallback: eligibility, relay, and fail-closed behaviour', () => { } /** Seeds and returns a non-terminal (`RUNNING`) run owned by the test's default token, so - * `DELETE /v2/actor-runs/:runId` throws `cannot-remove-running-run` - one of the error types `api.md`'s + * `DELETE /v2/actor-runs/:runId` throws `cannot-remove-running-run` - one of the error types the specification's * "Which local outcome each toggle covers" bullet lists as never relayed, regardless of either * toggle's state. */ async function seedRunningRun(): Promise { @@ -639,7 +639,7 @@ describe('api-fallback: eligibility, relay, and fail-closed behaviour', () => { }); // The response-header relay contract, pinned in both directions rather than left incidental - // (`api.md`'s "What a successful relay looks like"): `Set-Cookie` is the one repeated header name + // (the specification's "A successful relay" paragraph): `Set-Cookie` is the one repeated header name // this runtime's HTTP client can hand back as genuinely separate entries, and the one name where // comma-joining would corrupt the value (a cookie's own attributes routinely contain a comma, e.g. // `Expires=Wed, 21 Oct 2026 07:28:00 GMT`) - so it is always relayed as one line per cookie. Every @@ -682,7 +682,7 @@ describe('api-fallback: eligibility, relay, and fail-closed behaviour', () => { return res; } - /** `api.md`'s "Fail-closed guarantee" bullet requires the original local error to be reproduced + /** The specification's "Fail-closed guarantee" paragraph requires the original local error to be reproduced * unchanged - checked here as the header *name* set plus every value except `date`, whose value * legitimately differs run-to-run (its mere presence on both sides is asserted instead). This is * what would have caught the fail-closed mid-body-death path silently dropping diff --git a/test/integration/graceful-abort.test.ts b/test/integration/graceful-abort.test.ts index 3e33677a..db5251f9 100644 --- a/test/integration/graceful-abort.test.ts +++ b/test/integration/graceful-abort.test.ts @@ -1,5 +1,5 @@ /** - * `?gracefully=` abort contract (`requirements/api.md`'s "Graceful abort" section, + * `?gracefully=` abort contract (the abort entry in `src/api/openapi/actor-runtime.json`'s `x-actor-runtime-platform-notes`, * `GRACEFUL_ABORT_WINDOW_MS = 30000`): the `aborting` frame published before the fixed wait, * `driver.abortRun` withheld until the window elapses, the omitted/`false` path staying byte-identical to * an immediate abort, best-effort behavior with nobody connected, the READY-state and already-terminal @@ -146,7 +146,7 @@ describe('graceful abort (?gracefully=) contract', () => { await server.close(); }); - describe('graceful abort (?gracefully= contract per requirements/api.md "Graceful abort" section, GRACEFUL_ABORT_WINDOW_MS = 30000)', () => { + describe('graceful abort (?gracefully= contract per the platform notes in the runtime API specification, GRACEFUL_ABORT_WINDOW_MS = 30000)', () => { afterEach(() => { vi.useRealTimers(); }); @@ -219,7 +219,7 @@ describe('graceful abort (?gracefully=) contract', () => { const abortPromise = abortRun(driver, record, true); - // requirements/api.md's "Graceful abort" section: ABORTING lands immediately - observable well + // The specification's abort platform note: ABORTING lands immediately - observable well // before the 30s window elapses - and the aborting frame is published before the wait, not // after it. Waiting for the wait's own `setTimeout` to actually be scheduled is what proves both // already happened, since both come strictly before it in `abortRun`'s own code. @@ -432,7 +432,7 @@ describe('graceful abort (?gracefully=) contract', () => { // every other graceful-abort test in the "graceful abort" section above. const abortPromise = server.client.run(started.id).abort({ gracefully: true }); - // requirements/api.md's "Graceful abort" section: ABORTING lands immediately - observable over + // The specification's abort platform note: ABORTING lands immediately - observable over // the same real HTTP client, well before the HTTP response itself resolves. Polled in real time // (not via `waitForPendingTimer`): // `apify-client`'s own request pipeline can register an incidental `setTimeout` of its own before @@ -466,7 +466,7 @@ describe('graceful abort (?gracefully=) contract', () => { unsubscribe(); }); - it('POST .../abort with no gracefully parameter, over the same real HTTP round trip, still returns immediately with no wait (matches requirements/api.md\'s "omitted, or false" graceful-abort behavior end to end, not just at the service layer)', async () => { + it('POST .../abort with no gracefully parameter, over the same real HTTP round trip, still returns immediately with no wait (matches the specification\'s "omitted or false" graceful-abort behavior end to end, not just at the service layer)', async () => { const driver = deferredRunDriver(); server = await startTestServer(driver); const actor = await seedActor(server, 'immediate-http-actor'); diff --git a/test/integration/migration.test.ts b/test/integration/migration.test.ts index 3eb5a38a..cc85a405 100644 --- a/test/integration/migration.test.ts +++ b/test/integration/migration.test.ts @@ -1,5 +1,5 @@ /** - * Migration emulation and reboot (`requirements/api.md`, "Migration emulation"): the + * Migration emulation and reboot (the `migrateRun` operation in `src/api/openapi/actor-runtime.json`): the * `POST /actor-runtime/migrate/:runId` and `POST /v2/actor-runs/:runId/reboot` contracts. The console's * Migrate button is covered in `migrate-console.test.ts`. Fake-timer discipline follows * `graceful-abort.test.ts` (only `setTimeout`/`clearTimeout` faked). diff --git a/test/unit/actor-runtime-spec.test.ts b/test/unit/actor-runtime-spec.test.ts index 61681de0..294190fe 100644 --- a/test/unit/actor-runtime-spec.test.ts +++ b/test/unit/actor-runtime-spec.test.ts @@ -90,6 +90,22 @@ describe('the Actor runtime OpenAPI document', () => { } }); + it('lists the platform endpoints this runtime extends, so nothing local is left undescribed', () => { + const notes = (ACTOR_RUNTIME_OPENAPI as unknown as Record)[ + 'x-actor-runtime-platform-notes' + ] as Array>; + + expect(notes.map((note) => `${note.method} ${note.path}`)).toEqual([ + 'POST /v2/actors/{actorId}/runs', + 'POST /v2/actor-runs/{runId}/abort', + 'POST /v2/actor-runs/{runId}/reboot', + ]); + for (const note of notes) { + expect(note.summary).toBeTruthy(); + expect(note.description).toBeTruthy(); + } + }); + it('marks the events endpoint as the one websocket transport', () => { const websocketPaths = ACTOR_RUNTIME_OPERATIONS.filter((operation) => operation.transport === 'websocket').map( (operation) => operation.path, From 758f036a226536ff91d95b74ad6742d7e1b7651d Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 24 Sep 2026 11:37:26 +0000 Subject: [PATCH 3/3] Merge branch 'master' into claude/beautiful-noether-innrky #76 adds Actor Standby, and with it a new path inside this namespace: /actor-runtime/standby/