diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ab13977..1bd0f36 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -30,7 +30,8 @@ The sample Actors (`samples/actor_*`) crawl the live web, so running them needs The behavioural spec lives in `requirements/*.md`: `system.md`, `api.md`, `storage.md`, `actor-driver.md`, `cli.md`, `console.md` and `test.md`. `unsupported.md` lists the platform behaviour -the runtime deliberately leaves out. +the runtime deliberately leaves out. The one exception is the runtime-only `/actor-runtime/*` namespace, +whose contract is the OpenAPI document `src/api/openapi/actor-runtime.json`; `api.md` only points at it. User-facing documentation lives in `skills/actor-runtime/SKILL.md`, which ships inside the image. When a change alters what the runtime does for its users, update it in the same commit. Contribution diff --git a/README.md b/README.md index 1f01117..d952833 100644 --- a/README.md +++ b/README.md @@ -40,6 +40,16 @@ Your Actor's own output is passed through unchanged. - **Test pay-per-event pricing for free.** See what a run charges and costs, without spending money. - **Relay missing calls to the platform.** Send calls the runtime cannot answer to the real Apify API. +Most of these are driven by endpoints under `/actor-runtime/*`, which the real Apify API does not have. +The runtime describes that namespace in an OpenAPI document and serves it, so you never have to guess +what a given runtime supports - and, since the platform answers `404` there, the same call also tells you +whether you are pointed at a runtime at all: + +```bash +apify api GET /actor-runtime # every runtime-specific endpoint, its body and its responses +curl -s http://localhost:3333/actor-runtime/openapi.json | jq .paths # same document, for tooling +``` + ### Edit without rebuilding `apify push` registers the pushed folder as the Actor's **dev folder**. Every later run mounts it over diff --git a/requirements/api.md b/requirements/api.md index 8f26838..5676c02 100644 --- a/requirements/api.md +++ b/requirements/api.md @@ -23,7 +23,7 @@ JSON object, or that the schema rejects, naming every offending field; `400` `invalid-input-schema` when the Actor's own schema is not valid. Both messages match the Apify platform's. A build with no input schema accepts any body, unvalidated. -- Four endpoints are exceptions to the `{data}` envelope: +- Five endpoints are exceptions to the `{data}` envelope: - `GET /v2/logs/:buildOrRunId` (and its `actor-builds`/`actor-runs` aliases): the body is plain text, never `{data}`-wrapped, matching apify-client-js's `log().get()`. - `GET /v2/datasets/:datasetId/items` (and its `actor-runs/:runId/dataset/items` alias): the body is @@ -31,6 +31,9 @@ `x-apify-pagination-*` response headers, matching apify-client-js's pagination handling. - `GET /actor-runtime/events/:runId`: a websocket upgrade, not a JSON response at all - see "Actor runtime API" below. + - `GET /actor-runtime/openapi.json`: the runtime's own OpenAPI document, served bare so standard + OpenAPI tooling can consume the URL - see "Actor runtime API" below. The same document _is_ + `{data}`-enveloped at `GET /actor-runtime`, which is what the CLI reads. - `POST /v2/actor-runs/:runId/charge`: a bare `{}`, matching the platform. - `*At` timestamp fields are ISO-8601 strings. - Log content matches the Apify platform's log format: every log line starts with an ISO-8601 UTC @@ -50,6 +53,8 @@ # 501 vs 404 +- This section is about the emulated platform surface only. The `/actor-runtime/*` namespace has its + own specification and its own rule for what it does not describe - see "Actor runtime API" below. - Which endpoints answer `501` (unimplemented spec path) instead of `404` (off-spec path entirely) is decided from a fixed, built-in list of known Apify API v2 spec paths - nothing is fetched from `docs.apify.com` at runtime. See "Known differences from the Apify platform" in `storage.md` for the @@ -133,7 +138,9 @@ read/write/delete) just above, which this runtime does implement. Both paths answer `501`, not `404`. - All endpoints from the specification that do not have implementation must return response `501 Not Implemented` -- All endpoints not present in specification must return `404 Not Found` - **except** the `/actor-runtime/*` +- All endpoints not present in specification must return `404 Not Found` - **except** `/actor-runtime/*`, + which is not part of the Apify API at all and answers from its own specification instead ("Actor + runtime API" below) # Last-run shortcuts @@ -147,8 +154,8 @@ own responses apply, `501` included. - `v2/actor-tasks/:taskId/runs/last*` is not implemented (`unsupported.md`). - **One source per request**: an Actor that resolves locally is answered locally, including every later - miss; only a request naming an Actor unknown here is eligible for the upstream fallback (below), and - then as the caller's original request, which the platform resolves end to end. + miss; only a request naming an Actor unknown here is eligible for the upstream fallback, and then as + the caller's original request, which the platform resolves end to end. # Actor Standby @@ -159,189 +166,25 @@ # Actor runtime API -- `/actor-runtime/*` is the API controlling functions specific to the local Actor runtime -- **`GET /actor-runtime/skill`** (also at `/v2/actor-runtime/skill`) - serves this runtime's own Agent - Skill, `skills/actor-runtime/SKILL.md`, as markdown; unauthenticated, since it documents how to - authenticate here. -- **`POST /actor-runtime/dev-folder/:actorId`** - registers (or clears) the Actor's local dev folder for - the bind-mount feature (`actor-driver.md`). `:actorId` accepts the same forms as the rest of the API - (id, plain name, `username~name`). - - **Authenticated** the same way as every `/v2` route, and scoped to the caller's own Actors. - - **No build-first precondition** - registration works for an Actor that has never been built at all. - - **Request body**: a JSON string - the absolute path to set, or `""` to clear. - - **Response**: on success, `{ data: { localDevFolder } }` - the same value the console detail page - shows (`console.md`), doubling as the read-back this design has no separate `GET` for. - - **Error responses**, by rejection reason: - - `400` `invalid-request` - the body isn't a JSON string, or the string isn't a valid absolute - path. - - `400` `dev-folder-path-not-found` - the path does not exist on the host. - - `400` `dev-folder-not-a-directory` - the path exists but is not a directory. - - `400` `dev-folder-check-failed` - the path could not be verified, for any other reason. - - `503` `dev-folder-check-unavailable` - Docker itself is unreachable. - - `500` `internal-error` - an operational fault unrelated to the submitted path. -- The console's own dev-folder form (`console.md`) does **not** go through this endpoint - it posts to a - console-local, unauthenticated route on the console's own port - but the two surfaces accept and - reject exactly the same inputs with the same outcomes. -- **`POST /v2/actors/:actorId/runs?devFolder=false`** - runs from the built image alone, ignoring the - registered dev folder for that one run only; the registration itself is unchanged. Any other value, or - no parameter, means the default behaviour. -- **`POST /actor-runtime/debug/:actorId`** - sets (or clears) the Actor's persistent debug-mode toggle - (`actor-driver.md`'s "Debug mode" section). `:actorId` accepts the same forms as the rest of the API. - - **Authenticated** the same way as every `/v2` route, and scoped to the caller's own Actors. - - **No build-first precondition** - the toggle itself needs no build to exist. - - **Request body**: a strict JSON object with exactly these fields: - - `enabled` (required, boolean). - - `language` (optional, one of `"auto"` / `"node"` / `"python"`; defaults to `"auto"`). - - `port` (optional, integer `1024..65535`; absent means "use the resolved language's own - default port at run start" - never a stored literal). - Any other key present is rejected. Every accepted call fully replaces the prior state for that - Actor (never a partial merge) - a field the body omits resets to its own default, it does not keep - whatever a previous call set. `{"enabled": false}` clears the whole toggle, whatever else the body - names. - - **Response**: on success, `{ data: { localDebug } }`, where `localDebug` is `null` when debug mode - is off, or `{ language, port }` when on - `port` here is a nominal default (`5678`) for an - unresolved `language: "auto"`, purely for display; the port a given run actually publishes depends - on that run's own resolved language (`actor-driver.md`). Same doubles-as-read-back contract as the - dev-folder endpoint - no separate `GET`. - - **Error responses**: `400` `invalid-request` for every malformed body (not a JSON object, an - unknown field, a missing/non-boolean `enabled`, an invalid `language`, or a `port` outside - `1024..65535`) - no state change on rejection. - - Worked examples: - ``` - POST /actor-runtime/debug/ --body '{"enabled": true}' - -> { "data": { "localDebug": { "language": "auto", "port": 5678 } } } - - POST /actor-runtime/debug/ --body '{"enabled": true, "language": "node", "port": 9229}' - -> { "data": { "localDebug": { "language": "node", "port": 9229 } } } - - POST /actor-runtime/debug/ --body '{"enabled": false}' - -> { "data": { "localDebug": null } } - - POST /actor-runtime/debug/ --body '{"enabled": true, "prot": 9229}' - -> 400 invalid-request "Unknown field \"prot\" - allowed fields are \"enabled\", \"language\", \"port\"." - ``` - - The console's own debug-mode form (`console.md`) does **not** go through this endpoint - same - console-local, unauthenticated split as the dev-folder form - but both surfaces accept and reject - exactly the same inputs with the same outcomes. -- **`POST /actor-runtime/browser-view/:actorId`** - sets or clears the Actor's browser-view toggle - (`actor-driver.md`'s "Browser view" section). Authenticated and owner-scoped like every `/v2` route; no - build-first precondition. - - **Body**: `{ "enabled": boolean, "interactive"?: boolean }`, `interactive` defaulting to `false`. A call - fully replaces the prior state; `{"enabled": false}` clears it. Any other shape is `400 invalid-request`. - - **Response**: `{ data: { localBrowserView: { interactive } | null } }` - the read-back; there is no `GET`. -- **`GET /actor-runtime/events/:runId`** - a websocket upgrade, reachable at exactly this one path on - the fixed API port (`system.md`). It carries the run's platform events: `systemInfo` once a second - (`actor-driver.md`), a one-off `aborting`-plus-`persistState` pair under `?gracefully=` (below), and a - one-off `migrating` frame when a migration is triggered ("Migration emulation" below). Each frame is a - single text message, `{"name": "...", "data": {...}}`. - - The endpoint has no authentication. The run id in the path is the only thing it scopes on, and a - connection only ever receives that run's own frames; one run never sees another's. - - An unknown or already-terminal run id gets a completed upgrade followed immediately by a `1008` - close with a reason, never a non-101 HTTP status - the Python SDK treats a refused first connection - as fatal to the Actor. - - A connection to a live run stays open until the run ends, when the server closes it with `1000`. It - is never dropped while healthy, except that a graceful runtime shutdown terminates every open - connection along with the rest of the server. A migration/reboot restart is not the run ending: the - restarted container reconnects to the same path. - - The _periodic_ `persistState` is never sent over this channel; both SDKs generate it themselves. - The server sends `persistState` exactly once per graceful abort, alongside `aborting` (matching the - platform), and never alongside `migrating` (the SDKs synthesize that one). - -## Graceful abort (`?gracefully=`) - -- `POST /v2/actor-runs/:runId/abort` accepts an optional `?gracefully=` boolean. -- Omitted or `false`: the run aborts immediately. -- `true` on a running run: the record moves to `ABORTING` at once, an `aborting` frame plus a - `persistState {"isMigrating": false}` frame (in that order, matching the platform) are published on - the run's events channel, and the container is stopped 30 seconds later. The request responds as soon - as the run is `ABORTING`, not when the window ends. -- The window ends as soon as the run's container exits, finalising the run `ABORTED` then. -- `true` on a run with no container (still `READY`, or already terminal): behaves as if omitted. -- A second abort arriving during an open window: another `?gracefully=true` joins that window and neither - restarts it nor stops the container early; a non-graceful one cancels it and stops the container at once. -- An abort stops the container immediately: an Actor gets no wind-down time beyond the window above. - -## Migration emulation (`POST /actor-runtime/migrate/:runId`) and reboot - -A platform migration is not a run status: the run stays `RUNNING` while its container is killed and a -new one starts for the same run - same run id, env vars, and default storages, in-memory state gone. -This runtime emulates that observable experience on demand: - -- **`POST /actor-runtime/migrate/:runId`** (also at `/v2/actor-runtime/migrate/:runId`) - authenticated - like the rest of this namespace, scoped to the caller's own runs. The console's run detail view - exposes the same trigger as a Migrate button (`console.md`). - - Publishes a `migrating` frame (empty payload) on the run's events channel immediately, stops the - container 5 seconds later (the platform promises only "a few seconds"), then restarts the same - run. Status stays `RUNNING`; `startedAt`, `finishedAt`, `exitCode`, the default storage ids, and - the container env are unchanged. `stats.migrationCount` increments once per performed stop. - - Responds immediately with the run object (same shape as `abort`/`reboot`). A second call during - the open window joins it: same response, no second frame or window. - - Errors: unknown/foreign run `404` `record-not-found`; finished run `403` `job-finished`; - `READY`/`ABORTING` `400` `invalid-request`. - - The timeout budget is per run, not per container: a restarted container gets only the remaining - `timeoutSecs`. - - An abort (graceful or hard) landing during the window or restart wins: the run ends `ABORTED`, - never restarted. -- **`POST /v2/actor-runs/:runId/reboot`** - the real platform endpoint the SDKs call from their default - `migrating` handler. Stops and restarts the run's container immediately (no warning frame), cancels an - open migration window, and increments `stats.rebootCount`. A finished run is `403` `job-finished`; a - non-terminal run with no container (`READY`, `ABORTING`) gets the count bump but no restart. -- The run object's `stats` carries `migrationCount`, `rebootCount`, `restartCount`, and `resurrectCount` - (the latter two always `0` here), initialized to `0` at run creation like the platform. -- The run's log is cumulative across restarts, with a one-line marker between the incarnations' output. - -## Upstream fallback (opt-in, off by default, all HTTP methods) - -- Two independent booleans, `fallbackUnimplementedEnabled` and `fallbackNotFoundEnabled`, gate whether - a request this runtime cannot satisfy locally is instead relayed to the real Apify platform. Both - default to `false`, and a restart always brings both back to `false`, regardless of how they were - last set. Either can be on without the other; all four combinations are valid. -- **`GET /actor-runtime/api-fallback`** (also reachable at `/v2/actor-runtime/api-fallback`, like every - other endpoint in this namespace) returns - `{ "data": { "fallbackUnimplementedEnabled": , "fallbackNotFoundEnabled": , "upstreamBaseUrl": } }`. - `upstreamBaseUrl` is the platform this runtime would relay to (default `https://api.apify.com`, or the - value of `APIFY_UPSTREAM_API_BASE_URL` if set) - reported for visibility, but read-only: no request - body can change it. -- **`POST /actor-runtime/api-fallback`** (same two mounts) accepts a body naming either field, or both; - a field the body doesn't mention keeps its current value. The response is the same shape `GET` - returns, showing the state immediately after the change. - - **Authenticated** the same way as every other route in this namespace: no token is `401` - `user-not-authenticated`, with no state change. - - **Error responses**: a body that isn't a JSON object (a JSON array, scalar, or `null`), a body - present but empty (`{}`), a body containing a key other than the two above, or a body where a - present key's value isn't a boolean, is `400` `invalid-request`, with no state change. -- **Which local outcome each toggle covers** (exhaustive - every other error response is never - eligible, under any toggle combination): - - `fallbackUnimplementedEnabled` covers a request the runtime does not serve at all: a local `404` - or `501` response (see "501 vs 404" above). From the caller's point of view both mean "nothing - local answers this", so one toggle covers both. - - `fallbackNotFoundEnabled` covers a request that reaches a route this runtime does serve, but - whose specific record id doesn't exist locally (`record-not-found`, see "Response envelopes" - above). - - Every other error type - `invalid-request`, `invalid-input`, `invalid-input-schema`, - `user-not-authenticated`, `cannot-remove-running-run`, `deleting-unfinished-build`, any - `dev-folder-*` type, `internal-error` - is never relayed, regardless of either toggle's state. -- **All HTTP methods are eligible for both toggles, writes included**: a `POST`/`PUT`/`DELETE` that - would otherwise 404/501 locally is relayed exactly like a `GET` when its toggle is on - and, if the - platform accepts it, becomes a real write against the caller's real account. This is a deliberate - consequence of opting in, not an oversight. An eligible request reaches the platform at most once, so - a relayed write is never duplicated. -- **A successful relay** returns the platform's response status and body to the caller unchanged, - marked with two response headers: `x-actor-runtime-fallback: ` naming which platform - served it, and `x-actor-runtime-fallback-trigger: unimplemented` or `record-not-found` naming which - toggle let it through. Only a final `2xx` status counts as successful. -- **Fail-closed guarantee**: anything else - a non-`2xx` response, a timeout, or the platform being - unreachable - reproduces the exact response the caller would have gotten with both toggles off: the - original local error, unchanged, with neither marker header present. The platform's own status or - body is never surfaced to the caller. -- **Only the caller's own presented token is ever forwarded.** A relayed request's `Authorization` - header is always the exact bearer token the caller themselves sent on that request - never a - different or runtime-internal credential, and never sent at all for a request this runtime didn't - authenticate. Enabling either toggle therefore means the caller's own Apify token reaches the - configured `upstreamBaseUrl` on every eligible request; this is the risk being opted into. -- **Never splits one request between the two sources**: where one request resolves several records in - turn ("Last-run shortcuts" above), the first record decides where all of them come from. -- **Never enriches a call that already succeeds locally**: a collection/list endpoint (e.g. - `GET /v2/datasets`) that already returns `200` from local data never consults either toggle and never - gains platform objects. Fallback only ever resolves an otherwise-failing request; it does not make a - local listing "complete". +- `/actor-runtime/*` is the local-runtime-only API: the developer conveniences the Apify platform has no + counterpart for - live dev folder, debug mode, browser view, migration emulation, upstream API + fallback, the per-run events channel, and the path form of a standby address (above). +- **`src/api/openapi/actor-runtime.json` is the specification for all of it**, and it is normative: + every path, method, request body, response payload, error type and behaviour is stated there and + deliberately not restated here. It also carries, under `x-actor-runtime-platform-notes`, what this + runtime adds to a few otherwise faithful platform endpoints - `?devFolder=false` on run start, + `?gracefully=` on abort, reboot, and the Actor object's locally shaped `standbyUrl`. +- The runtime serves that document at `GET /actor-runtime` (`{data}`-enveloped, so `apify api` reads it) + and at `GET /actor-runtime/openapi.json` (bare, for OpenAPI tooling). Both are unauthenticated, so a + client can enumerate a runtime before it holds a token (`cli.md`). +- Every endpoint in the namespace is served at both `/actor-runtime/*` and `/v2/actor-runtime/*` - the + same routes, the second mount existing only because `apify api` builds every URL against a base that + already ends in `/v2`. Neither mount is part of the emulated Apify API, and nothing under either is + ever relayed upstream. +- A request under `/actor-runtime/*` that the document does not describe is answered from the document, + never from the emulated platform surface: `404` `not-found` for an undescribed path (the message names + `GET /actor-runtime`), `405` `method-not-allowed` with an `Allow` header for an undescribed method on a + described path, and `426` `upgrade-required` for a plain HTTP request to the events websocket path. +- The console's own dev-folder, debug-mode and browser-view forms (`console.md`) do not go through these + endpoints - they post to console-local, unauthenticated routes on the console's own port - but the two + surfaces accept and reject exactly the same inputs with the same outcomes. diff --git a/requirements/cli.md b/requirements/cli.md index e1ef41b..4192dc6 100644 --- a/requirements/cli.md +++ b/requirements/cli.md @@ -43,6 +43,19 @@ With neither source, `/users/me`'s `proxy` field is omitted and no `APIFY_PROXY_PASSWORD` is set on run containers - never a placeholder. +## Enumerating the runtime's own endpoints + +- The runtime-specific endpoints (`/actor-runtime/*`, `api.md`) have no counterpart on the Apify + platform, so a client cannot learn them from the platform's own OpenAPI specification. The runtime + therefore serves its own specification for that namespace, and the CLI reads it with a single stock + call: **`apify api GET /actor-runtime`** - one response enumerating every runtime-specific endpoint, + its request body, and its responses. +- No token is needed for that call, and the real Apify platform answers `404` on the same path, so it + doubles as "is the CLI pointed at a local Actor runtime, and which one?" - the document's + `info.version` is the runtime's own version. +- A client that asks for something in that namespace which the specification does not describe is told + so in those terms: `404` naming `GET /actor-runtime`, or `405` with an `Allow` header (`api.md`). + ## Supported commands (POC) - `apify push` - creates the Actor and Actor version from local source and triggers @@ -71,7 +84,7 @@ `apify-cli` fetches its actor-templates manifest from the internet. Every later push/call/log-stream/storage-access, and every build of an already-pulled base image, needs no outbound network access (see `system.md`'s offline-after-first-build note) - unless the opt-in - upstream API fallback is enabled (`api.md`, "Upstream fallback"), in which case an eligible local + upstream API fallback is enabled (`api.md`, and the `api-fallback` operation in the runtime API specification it references), in which case an eligible local miss makes one outbound request to the configured upstream instead of failing offline. - The bundled sample Actors crawl the live web (`https://crawlee.dev/` by default), so an `apify call` that runs one of them needs outbound network access from the Actor container even though the diff --git a/requirements/console.md b/requirements/console.md index 35a4eba..40bce0b 100644 --- a/requirements/console.md +++ b/requirements/console.md @@ -105,7 +105,7 @@ ## Settings page - Every page's header navigation includes a link to `/settings`, the one page for the upstream API - fallback toggles (`api.md`'s "Upstream fallback" section); the link itself shows each toggle's current + fallback toggles (specified by the runtime API specification `api.md` references); the link itself shows each toggle's current value. - `/settings` shows `fallbackUnimplementedEnabled`, `fallbackNotFoundEnabled`, and `upstreamBaseUrl` (the same values the API's toggle endpoint reports), plus a warning that enabling either toggle diff --git a/skills/actor-runtime/SKILL.md b/skills/actor-runtime/SKILL.md index 9392f4f..b1a5c7f 100644 --- a/skills/actor-runtime/SKILL.md +++ b/skills/actor-runtime/SKILL.md @@ -284,6 +284,11 @@ a later step misses. Only a call naming an Actor this runtime does not know is r `apify api GET v2/datasets/~my-results/items`. Names match case-insensitively. A bare name without `~` is an id, except for an Actor, where it is also tried as a name. Another user's resource is not found here - the runtime only ever shows your own. +- `apify api GET /actor-runtime` prints this runtime's own OpenAPI document - every `/actor-runtime/*` + endpoint above, and what it adds to the platform endpoints it extends. No token needed, and the real + Apify platform has no such endpoint, so a `404` there means you are talking to the platform rather + than to a runtime. `http://localhost:3333/actor-runtime/openapi.json` serves it unenveloped, for + OpenAPI tooling. - Or unauthenticated by URL: `http://localhost:3333/v2/datasets?token=TOKEN`. - The `runs/last` shortcuts address an Actor's newest run without knowing its id: `apify api GET v2/actors//runs/last`, and the same under `/log`, `/dataset/items`, diff --git a/src/api/actor-runtime-spec.ts b/src/api/actor-runtime-spec.ts new file mode 100644 index 0000000..4ed5b4b --- /dev/null +++ b/src/api/actor-runtime-spec.ts @@ -0,0 +1,115 @@ +/** + * The `/actor-runtime/*` namespace's own OpenAPI document (`openapi/actor-runtime.json`) and the + * lookups the server drives from it. + * + * The document - not this module, and not `requirements/api.md` - is the single source of truth for + * which runtime-specific endpoints exist: `routes/actor-runtime-spec.ts` serves it verbatim so a client + * can enumerate the namespace, and the namespace's terminal handler decides `404` vs `405` vs `426` + * from these lookups. Adding an endpoint therefore means adding it to the document; a test + * (`test/integration/actor-runtime-spec.test.ts`) fails if a documented HTTP operation has no route + * behind it, so the two cannot drift apart silently. + * + * This is the namespace-local counterpart of `spec-table.ts`, which does the same 501-vs-404 job for + * the emulated Apify `/v2` surface from a vendored snapshot of the platform's own spec. + */ +import rawDocument from './openapi/actor-runtime.json' with { type: 'json' }; + +/** Only the parts of OpenAPI this runtime actually reads - not a general-purpose OpenAPI model. */ +interface OperationObject { + operationId?: string; + summary?: string; + /** `"websocket"` marks an operation served by the HTTP upgrade handler rather than an Express route. */ + 'x-actor-runtime-transport'?: string; +} + +interface OpenApiDocument { + openapi: string; + info: { title: string; version: string; description?: string }; + paths: Record>; +} + +/** Lower-case, as OpenAPI spells them; anything else under a path item (`parameters`, `summary`, ...) + * is not an operation. */ +const HTTP_METHODS = ['get', 'put', 'post', 'delete', 'options', 'head', 'patch', 'trace'] as const; + +export const ACTOR_RUNTIME_OPENAPI = rawDocument as unknown as OpenApiDocument; + +export interface ActorRuntimeOperation { + /** Upper-case, matching `req.method`. */ + method: string; + /** The OpenAPI path template, e.g. `/actor-runtime/debug/{actorId}`. */ + path: string; + operationId: string; + summary: string; + /** `websocket` operations are upgraded by `events-ws.ts`, so no Express route ever matches them. */ + transport: 'http' | 'websocket'; +} + +function readOperations(document: OpenApiDocument): ActorRuntimeOperation[] { + const operations: ActorRuntimeOperation[] = []; + for (const [path, pathItem] of Object.entries(document.paths)) { + for (const method of HTTP_METHODS) { + const operation = pathItem[method] as OperationObject | undefined; + if (!operation) continue; + operations.push({ + method: method.toUpperCase(), + path, + operationId: operation.operationId ?? `${method}${path}`, + summary: operation.summary ?? '', + transport: operation['x-actor-runtime-transport'] === 'websocket' ? 'websocket' : 'http', + }); + } + } + return operations; +} + +export const ACTOR_RUNTIME_OPERATIONS: readonly ActorRuntimeOperation[] = readOperations(ACTOR_RUNTIME_OPENAPI); + +/** `/actor-runtime/events/` and `/actor-runtime/events` are the same path here; the empty trailing + * segment is not a segment. A bare `/actor-runtime` normalizes to no segments beyond the namespace. */ +function segmentsOf(path: string): string[] { + return path.split('/').filter(Boolean); +} + +/** Structural match, same rule as `spec-table.ts`: equal segment count, literal segments equal, + * `{param}` segments wildcard. */ +function pathMatches(template: string, requestPath: string): boolean { + const templateSegments = segmentsOf(template); + const requestSegments = segmentsOf(requestPath); + if (templateSegments.length !== requestSegments.length) return false; + return templateSegments.every( + (segment, i) => (segment.startsWith('{') && segment.endsWith('}')) || segment === requestSegments[i], + ); +} + +/** + * Every operation the document describes on the path template `requestPath` matches - empty when the + * document describes no such path at all. The namespace's terminal handler branches on this: empty is + * a `404` (off-spec path), non-empty with no matching method is a `405` (the `Allow` header is built + * from exactly these). + * + * `requestPath` is the full namespace path (`/actor-runtime/...`), with or without a trailing slash, + * and without a query string. + */ +export function actorRuntimeOperationsAtPath(requestPath: string): ActorRuntimeOperation[] { + return ACTOR_RUNTIME_OPERATIONS.filter((operation) => pathMatches(operation.path, requestPath)); +} + +/** + * The router-relative Express path for a documented operation - `/actor-runtime/events/{runId}` -> + * `/events/:runId` - so a route can be registered from the document instead of from a second, + * hand-maintained copy of the same path. + */ +export function routerPathOf(operation: ActorRuntimeOperation): string { + const relative = segmentsOf(operation.path) + .slice(1) + .map((segment) => (segment.startsWith('{') && segment.endsWith('}') ? `:${segment.slice(1, -1)}` : segment)) + .join('/'); + return `/${relative}`; +} + +/** The one operation the document describes for this method and path, if any. */ +export function matchActorRuntimeOperation(method: string, requestPath: string): ActorRuntimeOperation | undefined { + const upperMethod = method.toUpperCase(); + return actorRuntimeOperationsAtPath(requestPath).find((operation) => operation.method === upperMethod); +} diff --git a/src/api/openapi/actor-runtime.json b/src/api/openapi/actor-runtime.json new file mode 100644 index 0000000..758650e --- /dev/null +++ b/src/api/openapi/actor-runtime.json @@ -0,0 +1,938 @@ +{ + "openapi": "3.1.0", + "info": { + "title": "Apify local Actor runtime API", + "version": "0.1.0", + "summary": "The local-runtime-only endpoints that have no counterpart on the Apify platform.", + "description": "Everything under `/actor-runtime/*` is specific to this local Actor runtime: developer-convenience controls (live dev folder, debug mode, browser view, migration emulation, upstream API fallback, the per-run events channel) that the real Apify platform API does not have. The emulated subset of the platform API itself is *not* described here - that surface follows https://docs.apify.com/api/openapi.json.\n\nThis document is the normative specification for the namespace: `requirements/api.md` states only that it exists and references it, rather than repeating any of it. What this runtime adds to a handful of otherwise faithful *platform* endpoints is listed under `x-actor-runtime-platform-notes` at the root of this document, so a client enumerating it sees every local deviation in one place.\n\nThe runtime serves this document from itself, so a client can discover what a given runtime supports: `GET /actor-runtime` returns it in the API's `{data}` envelope (readable through `apify api GET /actor-runtime`), `GET /actor-runtime/openapi.json` returns it bare for OpenAPI tooling. Both are unauthenticated; every other operation here is authenticated like every `/v2` route and scoped to the caller's own Actors/runs, except the events websocket, which scopes on its run id alone. Every operation is served at both `/actor-runtime/*` and `/v2/actor-runtime/*` (see `servers`).\n\nA request under `/actor-runtime/*` that this document does not describe never reaches the emulated platform API and is never relayed upstream: an undescribed path answers `404 not-found`, a described path addressed with an undescribed method answers `405 method-not-allowed` with an `Allow` header, and a plain HTTP request to a websocket path answers `426 upgrade-required`.", + "license": { + "name": "Apache-2.0", + "identifier": "Apache-2.0" + } + }, + "servers": [ + { + "url": "/", + "description": "Canonical mount on the runtime's API port - 3333 unless `ACTOR_RUNTIME_API_PORT` moves it (see requirements/system.md). Every `:3333` in this document is that default." + }, + { + "url": "/v2", + "description": "Alias mount serving the exact same routes, so `apify api /actor-runtime/...` reaches them: the CLI builds every URL against a base that already ends in `/v2`." + } + ], + "security": [ + { + "bearerAuth": [] + }, + { + "tokenQuery": [] + } + ], + "x-actor-runtime-platform-extensions": [ + { + "method": "POST", + "path": "/v2/actors/{actorId}/runs", + "parameter": "devFolder", + "description": "`?devFolder=false` runs from the built image alone, ignoring the Actor's registered live dev folder for that one run; the registration itself is unchanged. Any other value, or no parameter, means the default behaviour. A runtime-only query parameter on an otherwise faithful platform endpoint, listed here so a client enumerating this document sees every local deviation in one place." + } + ], + "paths": { + "/actor-runtime": { + "get": { + "operationId": "getActorRuntimeSpecification", + "summary": "Enumerate the runtime-specific API", + "description": "Returns this OpenAPI document, `{data}`-enveloped like every other JSON response on this API, so `apify api GET /actor-runtime` prints it. Unauthenticated: the document is static, identical for every caller, and carries no user data - a client can therefore identify a local Actor runtime and enumerate its runtime-specific endpoints before it has a token.", + "security": [], + "responses": { + "200": { + "description": "The runtime-specific OpenAPI document.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { + "data": { + "$ref": "#/components/schemas/OpenApiDocument" + } + } + } + } + } + } + } + } + }, + "/actor-runtime/openapi.json": { + "get": { + "operationId": "getActorRuntimeOpenApiDocument", + "summary": "The bare OpenAPI document", + "description": "The same document as `GET /actor-runtime`, served without the `{data}` envelope so standard OpenAPI tooling can consume the URL directly. One of the documented exceptions to the envelope rule in `requirements/api.md`. Unauthenticated for the same reason.", + "security": [], + "responses": { + "200": { + "description": "The runtime-specific OpenAPI document, unenveloped.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/OpenApiDocument" + } + } + } + } + } + } + }, + "/actor-runtime/skill": { + "get": { + "operationId": "getActorRuntimeSkill", + "summary": "This runtime's own Agent Skill", + "description": "Serves `skills/actor-runtime/SKILL.md` - the shipped, agent-facing instructions for driving this runtime - from inside the running image, so an agent can load them without the repository checkout. Markdown by default; `?format=json` returns the `{data}`-enveloped `{ name, description, content }` parsed from the file's front matter plus its full body. Unauthenticated, because explaining how to authenticate against this runtime is one of the things the skill does: a caller must be able to read it before it holds a token. `500` `skill-unavailable` when the file is missing from the image - a deployment fault, not a bad request.", + "security": [], + "parameters": [ + { + "name": "format", + "in": "query", + "required": false, + "description": "`json` for the enveloped, parsed form. Any other value (or none) serves the raw markdown.", + "schema": { + "type": "string", + "enum": ["json"] + } + } + ], + "responses": { + "200": { + "description": "The skill, as markdown by default or as `{data}`-enveloped JSON under `?format=json`.", + "content": { + "text/markdown": { + "schema": { + "type": "string" + } + }, + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { + "data": { + "$ref": "#/components/schemas/SkillDocument" + } + } + } + } + } + }, + "500": { + "description": "`skill-unavailable` - the skill file is not present in this image.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + } + } + } + }, + "/actor-runtime/dev-folder/{actorId}": { + "post": { + "operationId": "setActorDevFolder", + "summary": "Register or clear an Actor's live dev folder", + "description": "Registers the host directory bind-mounted over the image's working directory on every subsequent run of this Actor, so source edits take effect without a rebuild (`requirements/actor-driver.md`). Submitting `\"\"` clears the registration. The path is checked when it is registered and again at every run start, so a folder deleted after registration fails the run rather than running against an empty directory; a single run can opt out with `?devFolder=false` (see `x-actor-runtime-platform-notes`).\n\nScoped to the caller's own Actors. There is no build-first precondition: registration works for an Actor that has never been built. The response body doubles as the read-back - there is deliberately no separate `GET`.\n\nThe console's own dev-folder form is a console-local, unauthenticated route on the console's port and does not go through this endpoint, but both surfaces accept and reject exactly the same inputs with the same outcomes.", + "parameters": [ + { + "$ref": "#/components/parameters/ActorId" + } + ], + "requestBody": { + "required": true, + "description": "A JSON string: the absolute path to register, or `\"\"` to clear.", + "content": { + "application/json": { + "schema": { + "type": "string" + }, + "examples": { + "register": { + "summary": "Register a folder", + "value": "/abs/path/to/src" + }, + "clear": { + "summary": "Clear the registration", + "value": "" + } + } + } + } + }, + "responses": { + "200": { + "description": "The registration after the change.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { + "data": { + "$ref": "#/components/schemas/DevFolderStatus" + } + } + } + } + } + }, + "400": { + "description": "Rejected, with no state change. Error `type` names the reason: `invalid-request` (the body is not a JSON string, or the string is not a valid absolute path), `dev-folder-path-not-found` (the path does not exist on the host), `dev-folder-not-a-directory` (the path exists but is not a directory), `dev-folder-check-failed` (the path could not be verified, for any other reason).", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "401": { + "$ref": "#/components/responses/Unauthenticated" + }, + "404": { + "$ref": "#/components/responses/RecordNotFound" + }, + "500": { + "description": "`internal-error` - an operational fault unrelated to the submitted path.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "503": { + "description": "`dev-folder-check-unavailable` - Docker itself is unreachable, so the path could not be checked.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + } + } + } + }, + "/actor-runtime/debug/{actorId}": { + "post": { + "operationId": "setActorDebugMode", + "summary": "Set or clear an Actor's debug-mode toggle", + "description": "While debug mode is on, every run of this Actor starts paused waiting for a debugger, with the attach address printed in the run's own log and the debug port published on `127.0.0.1` (`requirements/actor-driver.md`).\n\nScoped to the caller's own Actors; no build-first precondition. Every accepted call fully replaces the prior state - never a partial merge, so a field the body omits resets to its own default rather than keeping what a previous call set. `{\"enabled\": false}` clears the whole toggle, whatever else the body names. The response doubles as the read-back; there is no separate `GET`.\n\nThe console's own debug-mode form is a console-local, unauthenticated route and does not go through this endpoint, but both surfaces accept and reject exactly the same inputs with the same outcomes.", + "parameters": [ + { + "$ref": "#/components/parameters/ActorId" + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/DebugModeRequest" + }, + "examples": { + "enableAutoDetected": { + "summary": "Turn on, auto-detect the language", + "value": { + "enabled": true + } + }, + "enableNode": { + "summary": "Turn on for Node on an explicit port", + "value": { + "enabled": true, + "language": "node", + "port": 9229 + } + }, + "disable": { + "summary": "Turn off", + "value": { + "enabled": false + } + } + } + } + } + }, + "responses": { + "200": { + "description": "The toggle after the change.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { + "data": { + "$ref": "#/components/schemas/DebugStatus" + } + } + }, + "examples": { + "enabledAutoDetected": { + "value": { + "data": { + "localDebug": { + "language": "auto", + "port": 5678 + } + } + } + }, + "enabledNode": { + "value": { + "data": { + "localDebug": { + "language": "node", + "port": 9229 + } + } + } + }, + "disabled": { + "value": { + "data": { + "localDebug": null + } + } + } + } + } + } + }, + "400": { + "description": "`invalid-request`, with no state change: the body is not a JSON object, names an unknown field, misses `enabled` or gives it a non-boolean, sets an invalid `language`, or sets a `port` outside `1024..65535`. Example message for an unknown field: `Unknown field \"prot\" - allowed fields are \"enabled\", \"language\", \"port\".`", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "401": { + "$ref": "#/components/responses/Unauthenticated" + }, + "404": { + "$ref": "#/components/responses/RecordNotFound" + } + } + } + }, + "/actor-runtime/browser-view/{actorId}": { + "post": { + "operationId": "setActorBrowserView", + "summary": "Set or clear an Actor's browser-view toggle", + "description": "While browser view is on, every run of this Actor prints a viewer URL in its log - a live view of the display the Actor's browser draws on, served on the console's port (`requirements/actor-driver.md`). `interactive` also forwards mouse and keyboard input.\n\nScoped to the caller's own Actors; no build-first precondition. A call fully replaces the prior state; `{\"enabled\": false}` clears it. The response doubles as the read-back; there is no separate `GET`.", + "parameters": [ + { + "$ref": "#/components/parameters/ActorId" + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/BrowserViewRequest" + }, + "examples": { + "watch": { + "summary": "Watch only", + "value": { + "enabled": true + } + }, + "interact": { + "summary": "Watch and control", + "value": { + "enabled": true, + "interactive": true + } + }, + "disable": { + "summary": "Turn off", + "value": { + "enabled": false + } + } + } + } + } + }, + "responses": { + "200": { + "description": "The toggle after the change.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { + "data": { + "$ref": "#/components/schemas/BrowserViewStatus" + } + } + } + } + } + }, + "400": { + "description": "`invalid-request`, with no state change - any body other than `{\"enabled\": boolean, \"interactive\"?: boolean}`.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "401": { + "$ref": "#/components/responses/Unauthenticated" + }, + "404": { + "$ref": "#/components/responses/RecordNotFound" + } + } + } + }, + "/actor-runtime/migrate/{runId}": { + "post": { + "operationId": "migrateRun", + "summary": "Emulate a platform migration of a running run", + "description": "A platform migration is not a run status: the run stays `RUNNING` while its container is killed and a new one starts for the same run - same run id, env vars and default storages, in-memory state gone. This endpoint emulates that observable experience on demand.\n\nIt publishes a `migrating` frame (empty payload) on the run's events channel immediately, stops the container 5 seconds later (the platform promises only \"a few seconds\"), then restarts the same run. Status stays `RUNNING`; `startedAt`, `finishedAt`, `exitCode`, the default storage ids and the container env are unchanged, and `stats.migrationCount` increments once per performed stop. The timeout budget is per run, not per container: a restarted container gets only the remaining `timeoutSecs`. The run's log is cumulative across restarts, with a one-line marker between the incarnations' output.\n\nResponds immediately with the run object, the same shape `abort`/`reboot` return. A second call during the open window joins it: same response, no second frame and no second window. An abort (graceful or hard) landing during the window or the restart wins - the run ends `ABORTED`, never restarted.\n\nThe console's run detail view exposes the same trigger as a Migrate button. The platform's own `POST /v2/actor-runs/{runId}/reboot` is implemented too - see `x-actor-runtime-platform-notes`.", + "parameters": [ + { + "$ref": "#/components/parameters/RunId" + } + ], + "responses": { + "200": { + "description": "The run object, read back after the migration was scheduled.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { + "data": { + "$ref": "#/components/schemas/Run" + } + } + } + } + } + }, + "400": { + "description": "`invalid-request` - the run is non-terminal but has no container to migrate (`READY` or `ABORTING`).", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "401": { + "$ref": "#/components/responses/Unauthenticated" + }, + "403": { + "description": "`job-finished` - the run has already finished.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "404": { + "$ref": "#/components/responses/RecordNotFound" + } + } + } + }, + "/actor-runtime/standby/{standbyLabel}": { + "get": { + "operationId": "proxyStandbyRequest", + "summary": "Reach an Actor's Standby server through the runtime", + "description": "Forwards the request to one of the Actor's Standby runs, starting runs as the standby pool requires, and streams the Actor's own response back. This is the path-based half of the runtime's two standby addressings; the other is the platform-shaped host form `http://.localhost:3333/`, where the Actor owns `/` of its own origin as it does on `*.apify.actor`. Both are served on the API port, and an Actor object's `standbyUrl` names the one that fits the caller (see this document's `x-actor-runtime-platform-notes`).\n\n**Every method and every sub-path reach the Actor**, not just the `GET` this entry declares: whatever follows the label is the path the Actor's server sees, the request body streams through untouched, and a websocket upgrade is proxied too. The operation is declared once because the runtime forwards the request rather than interpreting it - the request and response shapes are the Actor's own, not this API's.\n\n**Authentication is the API's** - `Authorization: Bearer`, `x-apify-authorization` or `?token=` - and the token is forwarded to the Actor unchanged, so an Actor may use its caller's token itself. Only the token owner's own Actors are served; this runtime has no cross-tenant standby.\n\n**Errors this layer produces**, all before the Actor is reached: `400` `standby-not-enabled` (the Actor has no `actorStandby.isEnabled`), `400` `invalid-input`/`invalid-input-schema` (a standby run that passes the Actor's input cannot build one), `401` `user-not-authenticated`, `404` `record-not-found` (no such Actor of the caller's, or no build tagged for standby), `502` `standby-bad-gateway`, `503` `standby-run-failed`, `503` `standby-run-finished`, `504` `standby-run-not-ready`. None of them is ever relayed upstream: this router answers ahead of the API's body parser and of the upstream fallback entirely (`setApiFallbackState`).\n\nStandby itself - the pool, the per-run request limits, the idle timeout, the configuration on the Actor - is the platform's own feature, implemented faithfully; only the two addresses above are local.", + "parameters": [ + { + "$ref": "#/components/parameters/StandbyLabel" + } + ], + "responses": { + "200": { + "description": "The Actor's own response, relayed verbatim - status, headers and body are the Actor's. Any status the Actor returns reaches the caller unchanged; the codes below are this router's own." + }, + "400": { + "description": "`standby-not-enabled`, or `invalid-input`/`invalid-input-schema` when the Actor's input cannot be built for the standby run.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "401": { + "$ref": "#/components/responses/Unauthenticated" + }, + "404": { + "$ref": "#/components/responses/RecordNotFound" + }, + "502": { + "description": "`standby-bad-gateway` - the standby run did not answer.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "503": { + "description": "`standby-run-failed` - a standby run could not be started; or `standby-run-finished` - the run serving the request ended first.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "504": { + "description": "`standby-run-not-ready` - the run did not start listening on `ACTOR_STANDBY_PORT` in time.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + } + } + } + }, + "/actor-runtime/api-fallback": { + "get": { + "operationId": "getApiFallbackState", + "summary": "Read the upstream API fallback state", + "responses": { + "200": { + "$ref": "#/components/responses/ApiFallbackState" + }, + "401": { + "$ref": "#/components/responses/Unauthenticated" + } + }, + "description": "Reads both toggles and the platform they would relay to. The toggles' full contract is on the `POST` below." + }, + "post": { + "operationId": "setApiFallbackState", + "summary": "Turn upstream API fallback on or off", + "description": "Two independent toggles gate whether a request this runtime cannot satisfy locally is relayed to the real Apify platform instead of failing. Both default to `false`, and a restart always brings both back to `false`, regardless of how they were last set. Either can be on without the other; all four combinations are valid. This `POST` is a partial update: a field the body doesn't mention keeps its current value. `upstreamBaseUrl` is reported on every response for visibility but is read-only - no request body can change it.\n\n**Which local outcome each toggle covers** (exhaustive - every other response is never eligible, under any toggle combination):\n- `fallbackUnimplementedEnabled` covers a request the runtime does not serve at all: a local `404` or `501`. From the caller's point of view both mean \"nothing local answers this\", so one toggle covers both.\n- `fallbackNotFoundEnabled` covers a request that reaches a route this runtime does serve, but whose specific record id doesn't exist locally (`record-not-found`).\n- Every other error type - `invalid-request`, `invalid-input`, `invalid-input-schema`, `user-not-authenticated`, `cannot-remove-running-run`, `deleting-unfinished-build`, any `dev-folder-*` type, `internal-error` - is never relayed, whatever either toggle is set to. Neither is anything under `/actor-runtime/*` itself, which has no counterpart upstream, the standby addresses (`proxyStandbyRequest`) included - those are answered by a router that runs ahead of the fallback, so no `standby-*` error ever reaches it.\n\n**All HTTP methods are eligible for both toggles, writes included**: a `POST`/`PUT`/`DELETE` that would otherwise 404/501 locally is relayed exactly like a `GET` when its toggle is on - and, if the platform accepts it, becomes a real write against the caller's real account. This is a deliberate consequence of opting in, not an oversight. An eligible request reaches the platform at most once, so a relayed write is never duplicated.\n\n**A successful relay** returns the platform's response status and body to the caller unchanged, marked with two response headers: `x-actor-runtime-fallback: ` naming which platform served it, and `x-actor-runtime-fallback-trigger: unimplemented` or `record-not-found` naming which toggle let it through. Only a final `2xx` status counts as successful.\n\n**Fail-closed guarantee**: anything else - a non-`2xx` response, a timeout, or the platform being unreachable - reproduces the exact response the caller would have gotten with both toggles off: the original local error, unchanged, with neither marker header present. The platform's own status or body is never surfaced to the caller.\n\n**Only the caller's own presented token is ever forwarded.** A relayed request's `Authorization` header is always the exact bearer token the caller themselves sent on that request - never a different or runtime-internal credential, and never sent at all for a request this runtime didn't authenticate. Enabling either toggle therefore means the caller's own Apify token reaches the configured `upstreamBaseUrl` on every eligible request; this is the risk being opted into.\n\n**Never splits one request between the two sources**: where one request resolves several records in turn - `GET /v2/actors/{actorId}/runs/last*` resolves the Actor, then its newest run, then that run's log or storage - the first record decides where all of them come from. An Actor that resolves locally is answered locally, every later miss included; only a request whose first record is unknown here is eligible, and then as the caller's original request, which the platform resolves end to end.\n\n**Never enriches a call that already succeeds locally**: a collection endpoint (e.g. `GET /v2/datasets`) that already returns `200` from local data never consults either toggle and never gains platform objects. Fallback only ever resolves an otherwise-failing request; it does not make a local listing \"complete\".", + "requestBody": { + "required": true, + "description": "A JSON object naming either toggle, or both.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ApiFallbackPatch" + }, + "examples": { + "both": { + "value": { + "fallbackUnimplementedEnabled": true, + "fallbackNotFoundEnabled": true + } + }, + "one": { + "value": { + "fallbackNotFoundEnabled": true + } + } + } + } + } + }, + "responses": { + "200": { + "$ref": "#/components/responses/ApiFallbackState" + }, + "400": { + "description": "`invalid-request`, with no state change: the body is not a JSON object, is present but empty (`{}`), names a key other than the two toggles, or gives a present key a non-boolean value.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "401": { + "$ref": "#/components/responses/Unauthenticated" + } + } + } + }, + "/actor-runtime/events/{runId}": { + "get": { + "operationId": "connectRunEvents", + "summary": "The run's platform events websocket", + "description": "A websocket upgrade, not a JSON response - the channel the Apify SDKs consume as the platform events socket. It carries `systemInfo` once a second, a one-off `aborting`-plus-`persistState` pair on a graceful abort, and a one-off `migrating` frame when a migration is triggered. Each frame is a single text message, `{\"name\": \"...\", \"data\": {...}}`.\n\nUnauthenticated by decision: the run id in the path is the only thing it scopes on, and a connection only ever receives that run's own frames. An unknown or already-terminal run id gets a completed upgrade followed immediately by a `1008` close with a reason, never a non-101 HTTP status - the Python SDK treats a refused first connection as fatal to the Actor. A connection to a live run stays open until the run ends, when the server closes it with `1000`. A migration or reboot restart is not the run ending: the restarted container reconnects to the same path.\n\nThis operation is served by the HTTP server's upgrade handler rather than by a route, so a plain `GET` that is not a websocket handshake answers `426 upgrade-required`.", + "security": [], + "x-actor-runtime-transport": "websocket", + "parameters": [ + { + "$ref": "#/components/parameters/RunId" + } + ], + "responses": { + "101": { + "description": "Switching protocols - the websocket is open. Cross-run isolation is structural; an unknown or terminal run is closed with `1008` right after the upgrade completes." + }, + "426": { + "description": "`upgrade-required` - the request reached this path without a websocket handshake.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + } + } + } + } + }, + "components": { + "securitySchemes": { + "bearerAuth": { + "type": "http", + "scheme": "bearer", + "description": "The same per-token identity every `/v2` route uses: any non-empty token authenticates as that token's own user, created on first sighting." + }, + "tokenQuery": { + "type": "apiKey", + "in": "query", + "name": "token", + "description": "The same token as `bearerAuth`, passed as a query parameter instead." + } + }, + "parameters": { + "ActorId": { + "name": "actorId", + "in": "path", + "required": true, + "description": "The Actor's id, its plain `name`, or a `~name` (the caller's own), `username~name` or `userId~name` reference - the same forms every other endpoint on this API accepts. Names and usernames match case-insensitively. An empty name is `400` `invalid-request`; anything else that does not resolve, another user's Actor included, is `404` `record-not-found`.", + "schema": { + "type": "string" + } + }, + "RunId": { + "name": "runId", + "in": "path", + "required": true, + "description": "The run's id.", + "schema": { + "type": "string" + } + }, + "StandbyLabel": { + "name": "standbyLabel", + "in": "path", + "required": true, + "description": "The Actor's standby address: the platform's `--` label, or the Actor's id. Matched case-insensitively against the caller's own Actors only; anything else is `404` `record-not-found`.", + "schema": { + "type": "string" + } + } + }, + "responses": { + "Unauthenticated": { + "description": "`user-not-authenticated` - no token was presented. No state change.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "RecordNotFound": { + "description": "`record-not-found` - no such record, or it belongs to another user. Ownership scoping is not distinguishable from absence, by design.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ErrorResponse" + } + } + } + }, + "ApiFallbackState": { + "description": "The fallback state, as of immediately after the call.", + "content": { + "application/json": { + "schema": { + "type": "object", + "required": ["data"], + "properties": { + "data": { + "$ref": "#/components/schemas/ApiFallbackState" + } + } + } + } + } + } + }, + "schemas": { + "ErrorResponse": { + "type": "object", + "required": ["error"], + "description": "The error envelope every endpoint on this API shares. `type` is what clients branch on.", + "properties": { + "error": { + "type": "object", + "required": ["type", "message"], + "properties": { + "type": { + "type": "string" + }, + "message": { + "type": "string" + } + } + } + } + }, + "OpenApiDocument": { + "type": "object", + "description": "An OpenAPI 3.1 document - this one.", + "required": ["openapi", "info", "paths"], + "properties": { + "openapi": { + "type": "string" + }, + "info": { + "type": "object" + }, + "paths": { + "type": "object" + } + }, + "additionalProperties": true + }, + "DevFolderStatus": { + "type": "object", + "required": ["localDevFolder"], + "properties": { + "localDevFolder": { + "type": ["string", "null"], + "description": "The registered absolute host path, or `null` when no folder is registered. The same value the console's Actor detail page shows." + } + } + }, + "DebugModeRequest": { + "type": "object", + "required": ["enabled"], + "additionalProperties": false, + "properties": { + "enabled": { + "type": "boolean" + }, + "language": { + "type": "string", + "enum": ["auto", "node", "python"], + "default": "auto", + "description": "`auto` classifies the Actor's image at run start." + }, + "port": { + "type": "integer", + "minimum": 1024, + "maximum": 65535, + "description": "Absent means \"use the resolved language's own default port at run start\" (`9229` Node, `5678` Python) - never a stored literal." + } + } + }, + "DebugStatus": { + "type": "object", + "required": ["localDebug"], + "properties": { + "localDebug": { + "oneOf": [ + { + "type": "object", + "required": ["language", "port"], + "properties": { + "language": { + "type": "string", + "enum": ["auto", "node", "python"] + }, + "port": { + "type": "integer", + "description": "For an unresolved `auto`, a nominal default (`5678`) shown purely for display: the port a given run actually publishes depends on that run's own resolved language." + } + } + }, + { + "type": "null" + } + ], + "description": "`null` when debug mode is off." + } + } + }, + "BrowserViewRequest": { + "type": "object", + "required": ["enabled"], + "additionalProperties": false, + "properties": { + "enabled": { + "type": "boolean" + }, + "interactive": { + "type": "boolean", + "default": false, + "description": "Also forward mouse and keyboard input to the Actor's browser." + } + } + }, + "BrowserViewStatus": { + "type": "object", + "required": ["localBrowserView"], + "properties": { + "localBrowserView": { + "oneOf": [ + { + "type": "object", + "required": ["interactive"], + "properties": { + "interactive": { + "type": "boolean" + } + } + }, + { + "type": "null" + } + ], + "description": "`null` when browser view is off." + } + } + }, + "ApiFallbackPatch": { + "type": "object", + "minProperties": 1, + "additionalProperties": false, + "properties": { + "fallbackUnimplementedEnabled": { + "type": "boolean", + "description": "Relay a request nothing local answers (a local `404` or `501`)." + }, + "fallbackNotFoundEnabled": { + "type": "boolean", + "description": "Relay a request that reaches a route this runtime serves, but whose record id does not exist locally (`record-not-found`)." + } + } + }, + "ApiFallbackState": { + "type": "object", + "required": ["fallbackUnimplementedEnabled", "fallbackNotFoundEnabled", "upstreamBaseUrl"], + "properties": { + "fallbackUnimplementedEnabled": { + "type": "boolean" + }, + "fallbackNotFoundEnabled": { + "type": "boolean" + }, + "upstreamBaseUrl": { + "type": "string", + "format": "uri", + "description": "The platform this runtime would relay to (`https://api.apify.com` by default, or `APIFY_UPSTREAM_API_BASE_URL`). Reported for visibility, read-only: no request body can change it." + } + } + }, + "Run": { + "type": "object", + "description": "An Actor run object, the same shape the emulated platform endpoints (`POST /v2/actor-runs/{runId}/abort`, `.../reboot`) return. `stats` carries `migrationCount`, `rebootCount`, `restartCount` and `resurrectCount`.", + "required": ["id", "status"], + "properties": { + "id": { + "type": "string" + }, + "actId": { + "type": "string" + }, + "status": { + "type": "string" + }, + "stats": { + "type": "object" + } + }, + "additionalProperties": true + }, + "SkillDocument": { + "type": "object", + "description": "The skill's YAML front matter (`name`, `description`) alongside the whole markdown file.", + "required": ["name", "description", "content"], + "properties": { + "name": { + "type": "string", + "description": "The skill's `name` field; empty when the front matter is missing or malformed." + }, + "description": { + "type": "string", + "description": "The skill's `description` field; empty on the same terms as `name`." + }, + "content": { + "type": "string", + "description": "The complete markdown file, front matter included." + } + } + } + } + }, + "x-actor-runtime-platform-notes": [ + { + "method": "POST", + "path": "/v2/actors/{actorId}/runs", + "parameter": "devFolder", + "summary": "Run once from the built image alone", + "description": "`?devFolder=false` runs from the built image alone, ignoring the Actor's registered live dev folder for that one run; the registration itself is unchanged. Any other value, or no parameter, means the default behaviour. The run's own log records that it was used." + }, + { + "method": "POST", + "path": "/v2/actor-runs/{runId}/abort", + "parameter": "gracefully", + "summary": "Graceful abort, with the platform’s warning frames and grace period", + "description": "`POST /v2/actor-runs/{runId}/abort` accepts an optional `?gracefully=` boolean. Omitted or `false`: the run aborts immediately. `true` on a running run: the record moves to `ABORTING` at once, an `aborting` frame plus a `persistState {\"isMigrating\": false}` frame (in that order, matching the platform) are published on the run's events channel, and the container is stopped 30 seconds later. The request responds as soon as the run is `ABORTING`, not when the window ends. The window ends as soon as the run's container exits, finalising the run `ABORTED` then. `true` on a run with no container (still `READY`, or already terminal) behaves as if omitted. A second abort arriving during an open window: another `?gracefully=true` joins that window and neither restarts it nor stops the container early; a non-graceful one cancels it and stops the container at once. An abort stops the container immediately: an Actor gets no wind-down time beyond the window above." + }, + { + "method": "POST", + "path": "/v2/actor-runs/{runId}/reboot", + "summary": "Reboot, the endpoint the SDKs call from their own migrating handler", + "description": "The real platform endpoint the SDKs call from their default `migrating` handler, implemented here: it stops and restarts the run's container immediately (no warning frame), cancels an open migration window, and increments `stats.rebootCount`. A finished run is `403 job-finished`; a non-terminal run with no container (`READY`, `ABORTING`) gets the count bump but no restart. The run object's `stats` carries `migrationCount`, `rebootCount`, `restartCount` and `resurrectCount` (the latter two always `0` here), initialized to `0` at run creation like the platform." + }, + { + "method": "GET", + "path": "/v2/actors/{actorId}", + "summary": "`standbyUrl` names a local standby address, chosen by the caller", + "description": "An Actor with Standby enabled reports a `standbyUrl` that exists only here: `http://--.localhost:3333` for a client on the host, and `http://apify-api:3333/actor-runtime/standby/--` for a request arriving from an Actor container, which is picked from the request's own `Host` - the platform's value does not vary by caller. Clients that cannot resolve `*.localhost` use the same path form on `localhost`. Both addresses are the `proxyStandbyRequest` operation of this namespace; everything else about Actor Standby is the platform's own behaviour, implemented faithfully, except that only the Actor's owner is served." + } + ] +} diff --git a/src/api/routes/actor-runtime-spec.ts b/src/api/routes/actor-runtime-spec.ts new file mode 100644 index 0000000..5bed835 --- /dev/null +++ b/src/api/routes/actor-runtime-spec.ts @@ -0,0 +1,107 @@ +/** + * The `/actor-runtime/*` namespace's self-description and its terminal handler - the two halves of + * "the OpenAPI document is what the namespace *is*" (`api/actor-runtime-spec.ts`). + * + * `mountActorRuntimeSpec` serves the document itself and is deliberately registered *before* the + * namespace router's `auth()` (`server.ts`): the document is static, identical for every caller and + * carries no user data, so a client can identify a local Actor runtime and enumerate its + * runtime-specific endpoints before it has a token. + * + * `mountActorRuntimeUnmatched` is registered last, after every real route, and answers everything that + * fell through from the document alone. Without it those requests would reach the app-level catch-all + * in `server.ts`, which reads the *platform* spec table and can only ever call them plain `404`s. + */ +import type { Router } from 'express'; + +import { sendData, sendError } from '../envelope.js'; +import { h } from '../handler.js'; +import { + ACTOR_RUNTIME_OPENAPI, + ACTOR_RUNTIME_OPERATIONS, + actorRuntimeOperationsAtPath, + matchActorRuntimeOperation, + routerPathOf, +} from '../actor-runtime-spec.js'; + +/** Both mounts (`/actor-runtime` and `/v2/actor-runtime`) are the same router, so a request's path + * within the namespace is all that identifies it - `req.baseUrl` differs between the two and is + * deliberately not used. */ +function namespacePath(routerRelativePath: string): string { + return `/actor-runtime${routerRelativePath}`; +} + +export function mountActorRuntimeSpec(router: Router): void { + // `{data}`-enveloped, like every other JSON response on this API: `apify api GET /actor-runtime` + // goes through apify-client-js, which unwraps `data` and would otherwise print `undefined`. + router.get( + '/', + h(async (_req, res) => { + sendData(res, ACTOR_RUNTIME_OPENAPI); + }), + ); + + // The same document unenveloped, for OpenAPI tooling pointed straight at the URL - one of the + // documented exceptions to the envelope rule (`api.md`). + router.get( + '/openapi.json', + h(async (_req, res) => { + res.status(200).json(ACTOR_RUNTIME_OPENAPI); + }), + ); + + // Every websocket operation the document declares is served by the HTTP server's `upgrade` event + // (`events-ws.ts`), never by Express - so a request that reaches Express on one of those paths is a + // plain request where a handshake was expected. Registered here, from the document itself, rather + // than in the unmatched handler below, because those operations declare no security: answering a + // missing token with `401` would contradict the document the same request can read. + for (const operation of ACTOR_RUNTIME_OPERATIONS) { + // A websocket handshake is a `GET` by protocol, so that is the only method a websocket operation + // can be declared under; any other method on the same path falls through to the `405` below. + if (operation.transport !== 'websocket' || operation.method !== 'GET') continue; + const path = routerPathOf(operation); + router.get( + path, + h(async (req, res) => { + sendError( + res, + 426, + 'upgrade-required', + `${req.method} ${namespacePath(req.path)} is a websocket endpoint - upgrade required`, + ); + }), + ); + } +} + +export function mountActorRuntimeUnmatched(router: Router): void { + router.use((req, res) => { + const path = namespacePath(req.path); + + if (matchActorRuntimeOperation(req.method, path)) { + // Documented, yet no route matched it: a wiring bug, not a caller error. Answered the same + // way the platform surface answers a spec path with nothing behind it, and covered by a test + // that exercises every documented operation. + sendError(res, 501, 'not-implemented', `${req.method} ${path} is not implemented by this runtime`); + return; + } + + const allowed = actorRuntimeOperationsAtPath(path).map((entry) => entry.method); + if (allowed.length > 0) { + res.setHeader('Allow', allowed.join(', ')); + sendError( + res, + 405, + 'method-not-allowed', + `${req.method} is not allowed on ${path} - allowed: ${allowed.join(', ')}`, + ); + return; + } + + sendError( + res, + 404, + 'not-found', + `${req.method} ${path} is not part of the Actor runtime API - GET /actor-runtime lists every endpoint it has`, + ); + }); +} diff --git a/src/api/routes/api-fallback.ts b/src/api/routes/api-fallback.ts index 2acf6f6..ee7381a 100644 --- a/src/api/routes/api-fallback.ts +++ b/src/api/routes/api-fallback.ts @@ -1,5 +1,5 @@ /** - * `GET`/`POST /actor-runtime/api-fallback` (`api.md`'s "Upstream fallback" section) - mounted on the + * `GET`/`POST /actor-runtime/api-fallback` (specified by the `getApiFallbackState`/`setApiFallbackState` operations in `../openapi/actor-runtime.json`) - mounted on the * same `/actor-runtime` sub-router `dev-folder.ts` already registers on, so it shares that router's * single `auth()` registration (`server.ts`) rather than adding its own, and is served at both mounts * (`/actor-runtime/api-fallback` and `/v2/actor-runtime/api-fallback`) the same way the dev-folder route diff --git a/src/api/routes/debug-mode.ts b/src/api/routes/debug-mode.ts index 4edd787..3a57f7f 100644 --- a/src/api/routes/debug-mode.ts +++ b/src/api/routes/debug-mode.ts @@ -5,7 +5,8 @@ * * Canonical body is a strict JSON object: `{"enabled": true}` (defaults `language` to `"auto"`, no port * override), `{"enabled": true, "language": "node", "port": 9229}`, or `{"enabled": false}` to clear - - * every other shape (an unknown field included) is `400 invalid-request` (`api.md`). + * every other shape (an unknown field included) is `400 invalid-request` (the `setActorDebugMode` + * operation in `src/api/openapi/actor-runtime.json`). * * Ownership-scoped exactly like `dev-folder.ts`: `resolveActorParam`, so a caller can only ever toggle * debug mode for their own Actor. @@ -33,7 +34,7 @@ export function mountDebugMode(router: Router): void { if (result.kind !== 'ok') throw invalidRequest(result.message); // The response body doubles as the read-back - there is deliberately no separate `GET` for this - // yet, same as the dev-folder endpoint (`api.md`). + // yet, same as the dev-folder endpoint - neither is described in `../openapi/actor-runtime.json`. sendData(res, debugStatus(result.actor)); }), ); diff --git a/src/api/routes/dev-folder.ts b/src/api/routes/dev-folder.ts index e544bee..cb9e794 100644 --- a/src/api/routes/dev-folder.ts +++ b/src/api/routes/dev-folder.ts @@ -9,8 +9,9 @@ * only one of the two mounts ever matches a given request, that `auth()` still runs exactly once per * request either way. * - * Canonical body is a JSON string: `'"/abs/path"'` to set, `'""'` to clear (`api.md`). A JSON value that - * parses but isn't a string is rejected the same way a malformed body is. + * Canonical body is a JSON string: `'"/abs/path"'` to set, `'""'` to clear (the `setActorDevFolder` + * operation in `src/api/openapi/actor-runtime.json`). A JSON value that parses but isn't a string is + * rejected the same way a malformed body is. * * Ownership-scoped like every other Actor write on this API port: `resolveActorParam`, so a caller can * only ever register a dev folder for their own Actor. diff --git a/src/api/routes/migrate.ts b/src/api/routes/migrate.ts index 000f6c6..2ff78bf 100644 --- a/src/api/routes/migrate.ts +++ b/src/api/routes/migrate.ts @@ -1,6 +1,6 @@ /** * `POST /actor-runtime/migrate/:runId` - triggers an emulated migration of one run - * (`requirements/api.md`, "Migration emulation"). In the local-runtime-only namespace because the real + * (the `migrateRun` operation in `src/api/openapi/actor-runtime.json`). In the local-runtime-only namespace because the real * platform has no migrate API. */ import type { Router } from 'express'; diff --git a/src/api/server.ts b/src/api/server.ts index 5e22aa1..8ba9f99 100644 --- a/src/api/server.ts +++ b/src/api/server.ts @@ -19,6 +19,7 @@ import { mountDebugMode } from './routes/debug-mode.js'; import { mountBrowserView } from './routes/browser-view.js'; import { mountMigrate } from './routes/migrate.js'; import { mountApiFallback } from './routes/api-fallback.js'; +import { mountActorRuntimeSpec, mountActorRuntimeUnmatched } from './routes/actor-runtime-spec.js'; import { mountSkill } from './routes/skill.js'; import { standbyProxy } from './standby-proxy.js'; import { attemptFallback, type LocalError } from '../services/api-fallback.js'; @@ -53,20 +54,32 @@ export function createApiServer(deps: ApiServerDeps): Express { // module mounted on this router (`mountDevFolder`, `mountMigrate`, `mountApiFallback`) rather than each registering // its own - they are the same router instance, so a second registration would just run `auth()` // twice per request for no benefit. + // What the namespace contains is specified by `openapi/actor-runtime.json`, which the runtime serves + // from itself (`routes/actor-runtime-spec.ts`) and decides its own 404/405/426 from. // Registered ahead of the `auth()`-wrapped router below so Express answers `/skill` here and lets // every other `/actor-runtime/*` path fall through to it - see `routes/skill.ts` for why it is public. + // It is registered on its own router rather than on the one below because that one ends in a terminal + // handler (`mountActorRuntimeUnmatched`), which would answer `/skill` instead of letting it through. const actorRuntimePublic = express.Router(); mountSkill(actorRuntimePublic); app.use('/actor-runtime', actorRuntimePublic); app.use('/v2/actor-runtime', actorRuntimePublic); const actorRuntime = express.Router(); + // Registered before this router's `auth()` on purpose, so the namespace can be enumerated without a + // token - see `routes/actor-runtime-spec.ts`. Everything after `auth()` below is authenticated as + // usual. + mountActorRuntimeSpec(actorRuntime); actorRuntime.use(auth()); mountDevFolder(actorRuntime, deps); mountDebugMode(actorRuntime); mountBrowserView(actorRuntime); mountMigrate(actorRuntime, deps); mountApiFallback(actorRuntime); + // Last on this router: everything that matched no route above is answered from the namespace's own + // OpenAPI document rather than falling through to the app-level catch-all, which knows only the + // emulated platform surface. + mountActorRuntimeUnmatched(actorRuntime); app.use('/actor-runtime', actorRuntime); // Also served at `/v2/actor-runtime/*` - the *same* router instance, no duplicated route logic - solely // because `apify api`'s own URL-building hardcodes a `/v2`-suffixed base (`${baseUrl}/${endpoint}`, diff --git a/src/console/server.ts b/src/console/server.ts index d5a7b89..c8a10cf 100644 --- a/src/console/server.ts +++ b/src/console/server.ts @@ -13,7 +13,7 @@ * object's owner by username, or its Actor as `username~actorname` (`console.md`). The * dev-folder form, the debug-mode form, and the Migrate button all write cross-user the same way - a * deliberate deviation from the API's own strictly-owner-scoped writes, not an accident; the `/settings` - * form is runtime-global by nature (`api.md`'s "Upstream fallback" section), so ownership doesn't apply + * form is runtime-global by nature (`src/api/openapi/actor-runtime.json`'s `api-fallback` operations), so ownership doesn't apply * to it at all. */ import { createRequire } from 'node:module'; diff --git a/src/services/api-fallback.ts b/src/services/api-fallback.ts index acf09a6..d853941 100644 --- a/src/services/api-fallback.ts +++ b/src/services/api-fallback.ts @@ -1,5 +1,5 @@ /** - * Upstream API fallback (`api.md`'s "Upstream fallback" section): when a call locally misses - either + * Upstream API fallback (the `setApiFallbackState` operation in `src/api/openapi/actor-runtime.json`): when a call locally misses - either * because nothing in this runtime serves the path/method at all, or because it does but the specific * record id doesn't exist - and the matching toggle is on, the request is replayed against the real * Apify platform instead of failing, and a successful reply's status/body/headers are relayed back @@ -101,7 +101,7 @@ const EXCLUDED_RESPONSE_HEADERS = new Set([ export type FallbackTrigger = 'unimplemented' | 'record-not-found'; -/** The exhaustive mapping from a local error's `type` to the toggle that gates it (`api.md`). Every +/** The exhaustive mapping from a local error's `type` to the toggle that gates it (`openapi/actor-runtime.json`). Every * other error `type` - `invalid-request`, `user-not-authenticated`, `cannot-remove-running-run`, * `deleting-unfinished-build`, any `dev-folder-*` type, `internal-error` - is never eligible, `null`. */ function triggerForErrorType(type: string): FallbackTrigger | null { diff --git a/src/services/migrations.ts b/src/services/migrations.ts index 0bdbadb..7771efe 100644 --- a/src/services/migrations.ts +++ b/src/services/migrations.ts @@ -1,5 +1,5 @@ /** - * Emulated migrations and reboots (`requirements/api.md`, "Migration emulation"). A migration is not a + * Emulated migrations and reboots (the `migrateRun` operation in `src/api/openapi/actor-runtime.json`). A migration is not a * run status - the run keeps running while its container is replaced, so this module only tracks which * container stops must restart the run instead of finishing it. */ diff --git a/src/services/runs.ts b/src/services/runs.ts index 56d6fce..577441a 100644 --- a/src/services/runs.ts +++ b/src/services/runs.ts @@ -608,7 +608,8 @@ function cancelGracefulAbort(runId: string): boolean { * `gracefully` on a `RUNNING` run publishes the platform's `aborting` + `persistState` frame pair and * returns the `ABORTING` record straight away, leaving `GRACEFUL_ABORT_WINDOW_MS` to run in the * background; other states take the immediate path. A second concurrent graceful abort joins the window, - * a hard one cancels it - see `requirements/api.md`. + * a hard one cancels it - see the abort entry in `src/api/openapi/actor-runtime.json`'s + * `x-actor-runtime-platform-notes`. * * Both flags come from `onBeforeTransition`, read inside the same mutex-serialized write that performs * the transition: a preceding `get()` could observe a stale status, and only the hook can tell "this call diff --git a/test/integration/actor-runtime-spec.test.ts b/test/integration/actor-runtime-spec.test.ts new file mode 100644 index 0000000..ca7882a --- /dev/null +++ b/test/integration/actor-runtime-spec.test.ts @@ -0,0 +1,101 @@ +/** + * The runtime serves its own `/actor-runtime/*` OpenAPI document, and everything that document does + * not describe is answered from the document too (`requirements/api.md`, "Actor runtime API"). + * + * The load-bearing test here is "every documented operation is actually served": it is what keeps the + * document and the routes from drifting apart, in either direction - a route added without a + * document entry loses its 404/405 contract, a document entry added without a route answers `501`. + */ +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import axios, { type AxiosResponse } from 'axios'; + +import { startTestServer, type TestServerHandle } from './helpers/test-server.js'; +import { ACTOR_RUNTIME_OPERATIONS } from '../../src/api/actor-runtime-spec.js'; + +describe('Actor runtime API self-description', () => { + let server: TestServerHandle; + + beforeEach(async () => { + server = await startTestServer(); + }); + + afterEach(async () => { + await server.close(); + }); + + function call(method: string, path: string, withToken = true): Promise { + return axios.request({ + method, + url: `${server.baseUrl}${path}`, + headers: withToken ? { Authorization: `Bearer ${server.token}` } : {}, + validateStatus: () => true, + }); + } + + it('serves the document at GET /actor-runtime, enveloped, without a token', async () => { + const res = await call('get', '/actor-runtime', false); + expect(res.status).toBe(200); + expect(res.data.data.openapi).toMatch(/^3\.1\./); + expect(Object.keys(res.data.data.paths)).toContain('/actor-runtime/dev-folder/{actorId}'); + }); + + it('serves the bare document at GET /actor-runtime/openapi.json, for OpenAPI tooling', async () => { + const bare = await call('get', '/actor-runtime/openapi.json', false); + const enveloped = await call('get', '/actor-runtime'); + expect(bare.status).toBe(200); + expect(bare.data).not.toHaveProperty('data'); + expect(bare.data).toEqual(enveloped.data.data); + }); + + it('serves the document identically at both mounts', async () => { + const canonical = await call('get', '/actor-runtime'); + const alias = await call('get', '/v2/actor-runtime'); + expect(alias.status).toBe(200); + expect(alias.data).toEqual(canonical.data); + }); + + // One request per documented HTTP operation, with placeholder ids and no body: whatever the + // endpoint makes of that is its own business (`400`, `record-not-found`, ...), as long as the + // request reached a route at all rather than the namespace's unmatched handler. + const httpOperations = ACTOR_RUNTIME_OPERATIONS.filter((operation) => operation.transport === 'http'); + it.each(httpOperations.map((operation) => [`${operation.method} ${operation.path}`, operation] as const))( + '%s is served by a route, not by the unmatched handler', + async (_label, operation) => { + const path = operation.path.replace(/\{[^}]+\}/g, 'does-not-exist-id'); + const res = await call(operation.method.toLowerCase(), path); + expect(res.status).not.toBe(501); + expect(res.status).not.toBe(405); + expect(res.status).not.toBe(426); + if (res.status === 404) expect(res.data.error.type).toBe('record-not-found'); + }, + ); + + it('answers an undocumented path in the namespace with 404 not-found, at both mounts', async () => { + for (const path of ['/actor-runtime/does-not-exist', '/v2/actor-runtime/does-not-exist']) { + const res = await call('get', path); + expect(res.status).toBe(404); + expect(res.data.error.type).toBe('not-found'); + // The message points at the endpoint that lists what the namespace does have. + expect(res.data.error.message).toContain('GET /actor-runtime'); + } + }); + + it('answers an undocumented method on a documented path with 405 and an Allow header', async () => { + const res = await call('get', '/actor-runtime/debug/does-not-exist-id'); + expect(res.status).toBe(405); + expect(res.data.error.type).toBe('method-not-allowed'); + expect(res.headers.allow).toBe('POST'); + + const fallback = await call('put', '/actor-runtime/api-fallback'); + expect(fallback.status).toBe(405); + expect(fallback.headers.allow?.split(', ').sort()).toEqual(['GET', 'POST']); + }); + + it('answers a plain request to the events websocket with 426, with or without a token', async () => { + for (const withToken of [true, false]) { + const res = await call('get', '/actor-runtime/events/some-run', withToken); + expect(res.status).toBe(426); + expect(res.data.error.type).toBe('upgrade-required'); + } + }); +}); diff --git a/test/integration/api-fallback.test.ts b/test/integration/api-fallback.test.ts index f7019bf..fae1172 100644 --- a/test/integration/api-fallback.test.ts +++ b/test/integration/api-fallback.test.ts @@ -1,5 +1,5 @@ /** - * Covers the upstream API fallback (`api.md`'s "Upstream fallback" section, `services/api-fallback.ts`): + * Covers the upstream API fallback (the `api-fallback` operations in `src/api/openapi/actor-runtime.json`, `services/api-fallback.ts`): * the `GET`/`POST /actor-runtime/api-fallback` toggle-state endpoint, the eligibility mapping (both * toggles, in isolation and together), the fail-closed guarantee, own-token-only forwarding, and the two * marker headers - against a stubbed upstream (`helpers/stub-upstream.ts`, the pattern @@ -70,7 +70,7 @@ function startHeadersThenDieUpstream(): Promise { } /** A fixed `2xx` JSON response with a distinguishing header - the shape every successful-relay test - * below checks the caller receives unchanged (`api.md`'s "A successful relay" bullet). */ + * below checks the caller receives unchanged (the specification's "A successful relay" paragraph). */ function fixedOkResponse(distinguishingValue: string) { return () => ({ status: 200, @@ -312,7 +312,7 @@ describe('api-fallback: eligibility, relay, and fail-closed behaviour', () => { } /** Seeds and returns a non-terminal (`RUNNING`) run owned by the test's default token, so - * `DELETE /v2/actor-runs/:runId` throws `cannot-remove-running-run` - one of the error types `api.md`'s + * `DELETE /v2/actor-runs/:runId` throws `cannot-remove-running-run` - one of the error types the specification's * "Which local outcome each toggle covers" bullet lists as never relayed, regardless of either * toggle's state. */ async function seedRunningRun(): Promise { @@ -435,11 +435,13 @@ describe('api-fallback: eligibility, relay, and fail-closed behaviour', () => { process.env.APIFY_UPSTREAM_API_BASE_URL = stub.baseUrl; try { // Express's own default mount matching is case-insensitive, so this reaches the same - // `actor-runtime` sub-router as the lowercase path above, finds no matching PUT route, and - // falls through to the terminal catch-all with the original casing intact in `originalUrl`. + // `actor-runtime` sub-router as the lowercase path above and finds no matching PUT route. + // The namespace's own unmatched handler answers it from the runtime's OpenAPI document + // (`405`, since the path itself is documented for GET/POST) and never reaches the + // app-level catch-all - either way, nothing about it is eligible for relaying. const res = await call('put', '/v2/ACTOR-RUNTIME/api-fallback'); - expect(res.status).toBe(404); - expect(res.data.error.type).toBe('not-found'); + expect(res.status).toBe(405); + expect(res.data.error.type).toBe('method-not-allowed'); expect(res.headers['x-actor-runtime-fallback']).toBeUndefined(); expect(stub.hitCount()).toBe(0); } finally { @@ -662,7 +664,7 @@ describe('api-fallback: eligibility, relay, and fail-closed behaviour', () => { }); // The response-header relay contract, pinned in both directions rather than left incidental - // (`api.md`'s "What a successful relay looks like"): `Set-Cookie` is the one repeated header name + // (the specification's "A successful relay" paragraph): `Set-Cookie` is the one repeated header name // this runtime's HTTP client can hand back as genuinely separate entries, and the one name where // comma-joining would corrupt the value (a cookie's own attributes routinely contain a comma, e.g. // `Expires=Wed, 21 Oct 2026 07:28:00 GMT`) - so it is always relayed as one line per cookie. Every @@ -705,7 +707,7 @@ describe('api-fallback: eligibility, relay, and fail-closed behaviour', () => { return res; } - /** `api.md`'s "Fail-closed guarantee" bullet requires the original local error to be reproduced + /** The specification's "Fail-closed guarantee" paragraph requires the original local error to be reproduced * unchanged - checked here as the header *name* set plus every value except `date`, whose value * legitimately differs run-to-run (its mere presence on both sides is asserted instead). This is * what would have caught the fail-closed mid-body-death path silently dropping diff --git a/test/integration/graceful-abort.test.ts b/test/integration/graceful-abort.test.ts index 8906b44..8dcc752 100644 --- a/test/integration/graceful-abort.test.ts +++ b/test/integration/graceful-abort.test.ts @@ -1,5 +1,6 @@ /** - * `?gracefully=` abort contract (`requirements/api.md`'s "Graceful abort" section, + * `?gracefully=` abort contract (the abort entry in `src/api/openapi/actor-runtime.json`'s + * `x-actor-runtime-platform-notes`, * `GRACEFUL_ABORT_WINDOW_MS = 30000`): the `aborting` frame published before the fixed wait, * `driver.abortRun` withheld until the window elapses, the omitted/`false` path staying byte-identical to * an immediate abort, best-effort behavior with nobody connected, the READY-state and already-terminal @@ -160,7 +161,7 @@ describe('graceful abort (?gracefully=) contract', () => { await server.close(); }); - describe('graceful abort (?gracefully= contract per requirements/api.md "Graceful abort" section, GRACEFUL_ABORT_WINDOW_MS = 30000)', () => { + describe("graceful abort (?gracefully= contract per the abort entry in src/api/openapi/actor-runtime.json's x-actor-runtime-platform-notes, GRACEFUL_ABORT_WINDOW_MS = 30000)", () => { afterEach(() => { vi.useRealTimers(); }); @@ -231,7 +232,7 @@ describe('graceful abort (?gracefully=) contract', () => { // itself, only on `setTimeout`'s own virtual schedule. vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout'] }); - // requirements/api.md's "Graceful abort": the call answers as soon as the record is ABORTING and + // The specification's abort platform note: the call answers as soon as the record is ABORTING and // never blocks its caller (and so never holds an HTTP response open) for the 30s. const armed = await abortRun(driver, record, true); expect(armed?.status).toBe('ABORTING'); @@ -480,7 +481,7 @@ describe('graceful abort (?gracefully=) contract', () => { unsubscribe(); }); - it('POST .../abort with no gracefully parameter, over the same real HTTP round trip, still returns immediately with no wait (matches requirements/api.md\'s "omitted, or false" graceful-abort behavior end to end, not just at the service layer)', async () => { + it('POST .../abort with no gracefully parameter, over the same real HTTP round trip, still returns immediately with no wait (matches the specification\'s "omitted, or false" graceful-abort behavior end to end, not just at the service layer)', async () => { const driver = deferredRunDriver(); server = await startTestServer(driver); const actor = await seedActor(server, 'immediate-http-actor'); diff --git a/test/integration/migration.test.ts b/test/integration/migration.test.ts index 3eb5a38..cc85a40 100644 --- a/test/integration/migration.test.ts +++ b/test/integration/migration.test.ts @@ -1,5 +1,5 @@ /** - * Migration emulation and reboot (`requirements/api.md`, "Migration emulation"): the + * Migration emulation and reboot (the `migrateRun` operation in `src/api/openapi/actor-runtime.json`): the * `POST /actor-runtime/migrate/:runId` and `POST /v2/actor-runs/:runId/reboot` contracts. The console's * Migrate button is covered in `migrate-console.test.ts`. Fake-timer discipline follows * `graceful-abort.test.ts` (only `setTimeout`/`clearTimeout` faked). diff --git a/test/unit/actor-runtime-spec.test.ts b/test/unit/actor-runtime-spec.test.ts new file mode 100644 index 0000000..b57ab33 --- /dev/null +++ b/test/unit/actor-runtime-spec.test.ts @@ -0,0 +1,166 @@ +/** + * The `/actor-runtime/*` namespace's OpenAPI document is the source of truth for what the namespace + * contains (`src/api/actor-runtime-spec.ts`), so these guard the document itself and the lookups the + * server drives from it. That the documented operations are actually *served* is asserted against a + * running server in `test/integration/actor-runtime-spec.test.ts`. + */ +import { readFileSync } from 'node:fs'; +import { describe, expect, it } from 'vitest'; + +import { + ACTOR_RUNTIME_OPENAPI, + ACTOR_RUNTIME_OPERATIONS, + actorRuntimeOperationsAtPath, + matchActorRuntimeOperation, + routerPathOf, +} from '../../src/api/actor-runtime-spec.js'; + +describe('the Actor runtime OpenAPI document', () => { + it('is an OpenAPI 3.1 document describing only the /actor-runtime namespace', () => { + expect(ACTOR_RUNTIME_OPENAPI.openapi).toMatch(/^3\.1\./); + expect(Object.keys(ACTOR_RUNTIME_OPENAPI.paths).length).toBeGreaterThan(0); + for (const path of Object.keys(ACTOR_RUNTIME_OPENAPI.paths)) { + expect(path === '/actor-runtime' || path.startsWith('/actor-runtime/')).toBe(true); + } + }); + + it("carries the runtime's own version, so a client can tell two runtimes apart", () => { + const { version } = JSON.parse(readFileSync(new URL('../../package.json', import.meta.url), 'utf8')) as { + version: string; + }; + expect(ACTOR_RUNTIME_OPENAPI.info.version).toBe(version); + }); + + it('gives every operation a unique operationId and a summary', () => { + const operationIds = ACTOR_RUNTIME_OPERATIONS.map((operation) => operation.operationId); + expect(new Set(operationIds).size).toBe(operationIds.length); + for (const operation of ACTOR_RUNTIME_OPERATIONS) { + expect(operation.operationId).not.toBe(''); + expect(operation.summary).not.toBe(''); + } + }); + + it('describes every endpoint the namespace has', () => { + const described = ACTOR_RUNTIME_OPERATIONS.map((operation) => `${operation.method} ${operation.path}`).sort(); + expect(described).toEqual( + [ + 'GET /actor-runtime', + 'GET /actor-runtime/openapi.json', + 'GET /actor-runtime/skill', + 'GET /actor-runtime/api-fallback', + 'POST /actor-runtime/api-fallback', + 'POST /actor-runtime/browser-view/{actorId}', + 'POST /actor-runtime/debug/{actorId}', + 'POST /actor-runtime/dev-folder/{actorId}', + 'POST /actor-runtime/migrate/{runId}', + 'GET /actor-runtime/events/{runId}', + 'GET /actor-runtime/standby/{standbyLabel}', + ].sort(), + ); + }); + + it('declares at least one response per operation, and resolves every $ref it uses', () => { + const document = ACTOR_RUNTIME_OPENAPI as unknown as Record; + + for (const [path, pathItem] of Object.entries(ACTOR_RUNTIME_OPENAPI.paths)) { + for (const [method, operation] of Object.entries(pathItem)) { + const responses = (operation as { responses?: Record }).responses ?? {}; + expect(Object.keys(responses).length, `${method} ${path} declares no response`).toBeGreaterThan(0); + } + } + + // A typo'd `$ref` is the one way this document can be structurally broken while still parsing as + // JSON, and nothing else in the suite would notice. + const refsIn = (value: unknown): string[] => { + if (Array.isArray(value)) return value.flatMap(refsIn); + if (typeof value !== 'object' || value === null) return []; + return Object.entries(value).flatMap(([key, child]) => + key === '$ref' && typeof child === 'string' ? [child] : refsIn(child), + ); + }; + const resolve = (ref: string): unknown => + ref + .replace(/^#\//, '') + .split('/') + .reduce((node, segment) => (node as Record | undefined)?.[segment], document); + + const refs = refsIn(document); + expect(refs.length).toBeGreaterThan(0); + for (const ref of refs) { + expect(ref.startsWith('#/'), `${ref} is not a local reference`).toBe(true); + expect(resolve(ref), `${ref} does not resolve`).toBeDefined(); + } + }); + + it('lists the platform endpoints this runtime extends, so nothing local is left undescribed', () => { + const notes = (ACTOR_RUNTIME_OPENAPI as unknown as Record)[ + 'x-actor-runtime-platform-notes' + ] as Array>; + + expect(notes.map((note) => `${note.method} ${note.path}`)).toEqual([ + 'POST /v2/actors/{actorId}/runs', + 'POST /v2/actor-runs/{runId}/abort', + 'POST /v2/actor-runs/{runId}/reboot', + 'GET /v2/actors/{actorId}', + ]); + for (const note of notes) { + expect(note.summary).toBeTruthy(); + expect(note.description).toBeTruthy(); + } + }); + + it('marks the events endpoint as the one websocket transport', () => { + const websocketPaths = ACTOR_RUNTIME_OPERATIONS.filter((operation) => operation.transport === 'websocket').map( + (operation) => operation.path, + ); + expect(websocketPaths).toEqual(['/actor-runtime/events/{runId}']); + }); +}); + +describe('matchActorRuntimeOperation', () => { + it('matches a documented operation through its {param} segments', () => { + expect(matchActorRuntimeOperation('POST', '/actor-runtime/debug/abc123')?.operationId).toBe( + 'setActorDebugMode', + ); + expect(matchActorRuntimeOperation('post', '/actor-runtime/debug/abc123')?.operationId).toBe( + 'setActorDebugMode', + ); + }); + + it('matches the namespace root with or without a trailing slash', () => { + expect(matchActorRuntimeOperation('GET', '/actor-runtime')?.operationId).toBe('getActorRuntimeSpecification'); + expect(matchActorRuntimeOperation('GET', '/actor-runtime/')?.operationId).toBe('getActorRuntimeSpecification'); + }); + + it('is sensitive to the method', () => { + expect(matchActorRuntimeOperation('GET', '/actor-runtime/debug/abc123')).toBeUndefined(); + }); + + it('is sensitive to the segment count, so no path is matched by a prefix of it', () => { + expect(matchActorRuntimeOperation('POST', '/actor-runtime/debug')).toBeUndefined(); + expect(matchActorRuntimeOperation('POST', '/actor-runtime/debug/abc123/extra')).toBeUndefined(); + }); + + it('does not match an undocumented path at all', () => { + expect(matchActorRuntimeOperation('GET', '/actor-runtime/does-not-exist')).toBeUndefined(); + expect(actorRuntimeOperationsAtPath('/actor-runtime/does-not-exist')).toEqual([]); + }); + + it('reports every method documented on a path, which is what the Allow header is built from', () => { + expect( + actorRuntimeOperationsAtPath('/actor-runtime/api-fallback') + .map((entry) => entry.method) + .sort(), + ).toEqual(['GET', 'POST']); + }); +}); + +describe('routerPathOf', () => { + it('turns a documented path template into the Express path the router registers', () => { + const events = matchActorRuntimeOperation('GET', '/actor-runtime/events/some-run')!; + expect(routerPathOf(events)).toBe('/events/:runId'); + + const root = matchActorRuntimeOperation('GET', '/actor-runtime')!; + expect(routerPathOf(root)).toBe('/'); + }); +});