diff --git a/CHANGELOG.md b/CHANGELOG.md index acbadf8e..4fb142d5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,13 @@ product overview. ## Unreleased +- Fix a history-transfer deadlock when `/sudo` crosses a container whose mounts need recreation. + Transfer preparation now excludes its own lease, reports waits, handles Stop, and times out + into transcript recovery instead of hanging indefinitely. +- Google Chat now reports a fatal Pub/Sub receiver error as a failed connection, stops intake, + and gives an actionable Subscriber-role hint for permission errors. Reconnect restores status + without allowing an old connection's late error to overwrite it. + - Channel Runtime now shows only engine and model controls for the shared login. Dedicated Codex sign-in uses a method dropdown: choosing ChatGPT starts the device flow and clicking its code copies it; the API key field appears only for that method. The shared sign-in moved to Settings. diff --git a/FEATURES.md b/FEATURES.md index eb06a7ba..1c141a23 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -1,5 +1,16 @@ # ChannelGate — Features +## Runtime history transfer and Google Chat failure recovery + +- Switching a thread between its container and `/sudo` host carries native history without + treating the transfer's own lease as an active occupant that blocks container recreation. + Preparation reports waits, responds to Stop, and falls back to transcript recovery after a + bounded preparation timeout. An explicit Stop never starts a replacement engine turn. +- Fatal Google Chat receiver errors stop intake and mark the connection as failed; health and + connector availability no longer report a stopped receiver as connected. Pub/Sub permission + failures name the Subscriber role and reconnect action. Reconnecting clears the failure, and + callbacks from older connections cannot overwrite the replacement connection's state. + ## Channel-specific Codex authentication - Runtime settings ask which Codex login a channel uses. The shared choice shows the channel's diff --git a/TEST-PLAN.md b/TEST-PLAN.md index 49794e35..c450d2df 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -1,5 +1,40 @@ # ChannelGate — Test Plan +## Runtime history transfer and Google Chat failure recovery (2026-10-02) + +- [x] Automated: `test/session-carry.test.js`, `test/container-carry.test.js`, + `test/container-lifecycle.test.js`, and `test/runtime-integration-run.test.js` cover caller-lease + forwarding through container→host transfer, stale mount recreation, lease release, Stop during + preparation, timeout fallback, and cancellation without starting a healed turn. +- [x] Automated: `test/platform-googlechat.test.js` and `test/platform-transport-manager.test.js` + cover a real puller receiving a simulated 403 after startup, stopped inbox dispatch, failed + connection status and absent live connector, reconnect recovery, early failure during startup, + and stale failure callbacks after reconnect/disconnect. This transport behavior is engine-independent. +- [x] Live container fixture proof: `CG_LIVE_CONTAINER=1 node --test + test/session-carry.live.test.js` passed with real Podman and the runtime image. Disposable + Claude and Codex native-layout fixture files roundtripped host→container→host; changing the + clean-workspace bind source recreated the container while its carry lease was held. Bytes + survived both rebuilds, leases returned to zero, and the disposable container/volume were + removed. This verifies file movement and lifecycle, not real-engine conversation acceptance. +- [ ] Live Claude and Codex acceptance (separate runs): use a disposable private admin test + channel with the authorized admin test actor, pin the engine, and send “Remember carry-check-42 + for the next message.” After its reply, change a fixture workspace mount without destroying its + persistent HOME; enable `/sudo` in the same thread and ask “What marker did I give you?”. Pass: + container recreation and native history transfer finish, the answer contains the marker, and + no indefinite “waiting for active runs” occurs. Repeat host→container after `/sudo off`. + With another fixture occupant holding a lease, repeat and press Stop during the preparation + wait. Pass: cancellation releases the transfer lease, starts no engine, and the next turn runs. + Copy commands remain bounded by their existing 120-second CLI timeout; cancellation during an + already executing copy is checked on completion, whereas readiness waits cancel immediately. +- [ ] Live Google Chat acceptance (engine-independent): use a dedicated test subscription and + service account without Subscriber, connect, then poll settings/health. Pass: status becomes + error with a Subscriber/reconnect remedy and no live connector remains. Grant Subscriber on + that subscription, reconnect, and send one test message; pass: exactly one reply. A separate + healthy Slack fixture must continue answering throughout. +- [ ] Private QA catalog: mirror the preceding two engine cases and the engine-independent + Google case to the requesting maintainer's selected personal Airtable connection. Pending + explicit connection-use authorization; automated results are not a claim of live acceptance. + ## Channel-specific Codex authentication (2026-09-29) - [x] Automated: Chromium confirms the shared-login channel view has no gateway status/sign-in diff --git a/src/gateway/run.js b/src/gateway/run.js index c1f35a1a..f9aff60a 100644 --- a/src/gateway/run.js +++ b/src/gateway/run.js @@ -979,7 +979,10 @@ export async function runMessage({ channelId, authorId, workspaceId = "", text, let carried = null; try { carried = await carrySession({ - engine, sessionId, cwd, storedRuntime: sessionRuntime, target, slug: entry.slug, threadKey, + engine, sessionId, cwd, storedRuntime: sessionRuntime, target, slug: entry.slug, threadKey, signal, + announce: (text) => { + try { onEvent?.({ kind: "notice", scope: "gateway", text }); } catch { /* best-effort progress */ } + }, }); } finally { clearTimeout(carryTimer); diff --git a/src/gateway/session-carry.js b/src/gateway/session-carry.js index ae94ec16..43fb226a 100644 --- a/src/gateway/session-carry.js +++ b/src/gateway/session-carry.js @@ -59,6 +59,8 @@ export function buildCarryEntries({ engine, cwd, sessionId, fromDir, toDir }) { * @param {string} [options.threadKey] for the log line * @param {Function} [options.resolveFor] resolveRuntime, injectable for tests * @param {Function} [options.log] console.log, injectable for tests + * @param {AbortSignal} [options.signal] cancellation of the turn, including container preparation + * @param {Function} [options.announce] progress while the source or destination container is prepared * @returns {Promise<{direction: string, files: number}|null>} null when nothing was carried */ export async function carrySession({ @@ -71,6 +73,8 @@ export async function carrySession({ threadKey = "", resolveFor = resolveRuntime, log = console.log, + signal = null, + announce = null, } = {}) { const where = `${slug}/${threadKey}`; const to = String(target?.backend || "host"); @@ -79,6 +83,7 @@ export async function carrySession({ if (!sessionId || !target || from === to) return null; try { + signal?.throwIfAborted(); if (!engineSessionState(engine)) { log(`[gateway] ${where}: ${engine} does not declare where it keeps a session, so its history stays on ${from} — the resume falls back to the existing heal`); return null; @@ -106,16 +111,20 @@ export async function carrySession({ let result; try { result = to === "host" - ? await mover.runtime.copyOut(mover, entries) - : await mover.runtime.copyIn(mover, entries); + ? await mover.runtime.copyOut(mover, entries, { lease, signal, announce }) + : await mover.runtime.copyIn(mover, entries, { lease, signal, announce }); } finally { lease.release(); } + signal?.throwIfAborted(); const files = Number(result?.copied) || 0; if (!files) return null; log(`[gateway] ${where}: carried ${engine} session ${sessionId} ${direction} (${files} file${files === 1 ? "" : "s"})`); return { direction, files }; } catch (error) { + // A user Stop must end the turn, never fall through to a fresh engine attempt. + signal?.throwIfAborted(); + if (error?.name === "AbortError") throw error; log(`[gateway] ${where}: session carry-over failed (${error?.message || error}) — the resume falls back to the existing heal`); return null; } diff --git a/src/platforms/boot.js b/src/platforms/boot.js index 4e76cd85..1c61500b 100644 --- a/src/platforms/boot.js +++ b/src/platforms/boot.js @@ -21,7 +21,7 @@ import { startTeams, botIdFor } from "./msteams/transport.js"; // The connector is built BEFORE the transport starts, on purpose: a Pub/Sub pull can deliver its // first event inside start(), and an ingest that does not exist yet would drop that message. Wiring // it up front means the very first message is answerable. -async function startGoogleChatTransport(config, log) { +async function startGoogleChatTransport(config, log, { onFatal } = {}) { const capabilities = requirePlatform("googlechat").capabilities; const auth = createGoogleAuth({ serviceAccount: config.serviceAccountJson }); const api = createChatApi({ auth }); @@ -31,6 +31,7 @@ async function startGoogleChatTransport(config, log) { ...config, capabilities, onMessage: ingest, + onFatal, log, deps: { auth, api, connector }, }); @@ -49,7 +50,7 @@ async function startTeamsTransport(config, log) { } export function createPlatformTransports({ log = console } = {}) { - const googlechat = createTransportManager({ platform: "googlechat", log, start: (config) => startGoogleChatTransport(config, log) }); + const googlechat = createTransportManager({ platform: "googlechat", log, start: (config, lifecycle) => startGoogleChatTransport(config, log, lifecycle) }); const msteams = createTransportManager({ platform: "msteams", log, start: (config) => startTeamsTransport(config, log) }); registerTransport("googlechat", googlechat); registerTransport("msteams", msteams); diff --git a/src/platforms/googlechat/pubsub.js b/src/platforms/googlechat/pubsub.js index 0c4384e0..b25cce0c 100644 --- a/src/platforms/googlechat/pubsub.js +++ b/src/platforms/googlechat/pubsub.js @@ -72,7 +72,12 @@ export function createPubSubPuller({ }); if (!res.ok) { const text = await res.text().catch(() => ""); - const err = new Error(`Pub/Sub ${path} failed (${res.status}): ${text.slice(0, 300)}`); + let detail = text.slice(0, 300); + try { detail = String(JSON.parse(text).error?.message || detail).slice(0, 300); } catch { /* non-JSON response */ } + const remedy = res.status === 403 + ? ` Check that the configured service account has Pub/Sub Subscriber on ${subscription}, then reconnect Google Chat.` + : ""; + const err = new Error(`Pub/Sub ${path} failed (${res.status}): ${detail}${remedy}`); err.status = res.status; // 401/403/404 are configuration facts, not weather: a revoked key, a service account without // roles/pubsub.subscriber, or a subscription that does not exist. Backing off and retrying diff --git a/src/platforms/googlechat/transport.js b/src/platforms/googlechat/transport.js index e3e27442..72ada4bf 100644 --- a/src/platforms/googlechat/transport.js +++ b/src/platforms/googlechat/transport.js @@ -14,6 +14,7 @@ export async function startGoogleChat({ capabilities, onMessage, onBotUserId = null, + onFatal = null, log = console, deps = {}, } = {}) { @@ -62,8 +63,12 @@ export async function startGoogleChat({ return inbox.accept({ id, conversationId, payload: { event, botId: selfId } }); } - const puller = deps.puller || createPubSubPuller({ auth, subscription, onEvent, log, - onFatal: (err) => log.error?.(`[googlechat] transport stopped: ${err?.message || err}`) }); + const puller = deps.puller || createPubSubPuller({ auth, subscription, onEvent, log, fetchImpl: deps.fetchImpl, + onFatal: (err) => { + inbox.stop(); + log.error?.(`[googlechat] transport stopped: ${err?.message || err}`); + onFatal?.(err); + } }); inbox.start(); try { puller.start(); } catch (error) { inbox.stop(); throw error; } diff --git a/src/platforms/manager.js b/src/platforms/manager.js index c6667de1..2a8604de 100644 --- a/src/platforms/manager.js +++ b/src/platforms/manager.js @@ -57,13 +57,22 @@ export function createTransportManager({ platform, start, log = console } = {}) error = null; await stopTransport(previous); try { - const started = await start(config); + // Fatal errors can arrive during start() or after it resolves. Bind them to + // this generation so a late failure from an old connection cannot poison its replacement. + let failed = false; + const onFatal = (err) => { + if (gen !== generation) return; + failed = true; + status = "error"; + error = err?.message || String(err); + }; + const started = await start(config, { onFatal }); if (gen !== generation) { await stopTransport(started); return snapshot(); } current = started; - status = "connected"; + if (!failed) status = "connected"; } catch (err) { log.error?.(`[${platform}] connect failed: ${err?.message || err}`); if (gen === generation) { @@ -90,7 +99,7 @@ export function createTransportManager({ platform, start, log = console } = {}) connect, disconnect, snapshot, - getConnector: () => current?.connector ?? null, + getConnector: () => status === "connected" ? current?.connector ?? null : null, // The live transport, for the few things that need more than a connector (the Teams webhook // handler the Express route mounts, the Chat puller's health). getTransport: () => current, diff --git a/src/runtimes/container/carry.js b/src/runtimes/container/carry.js index 5d8ddb44..f536d585 100644 --- a/src/runtimes/container/carry.js +++ b/src/runtimes/container/carry.js @@ -137,12 +137,29 @@ export function createContainerCarry({ exec, lifecycle, log = () => {} } = {}) { // A carry may run against a container the idle reaper has stopped — that is rather the point: // the history lives in a volume that outlives the container. Bringing it up is the backend's own // job (the caller holds a lease around the whole carry so the reaper cannot stop it again - // mid-copy). `retry` keeps the out-of-band-removal self-heal every other exec has. - async function runScript(target, script) { - await lifecycle.ensureUp(target, {}); - return exec.runExec(target, [target.container.name, "/bin/sh", "-c", script], { + // mid-copy). Forward that lease so a mount rebuild never waits on its own caller. + async function runScript(target, script, { lease = null, signal = null, announce = null } = {}) { + signal?.throwIfAborted(); + // A busy old container must not strand history transfer indefinitely. Timeout is a + // recoverable carry failure; an explicit Stop remains cancellation of the whole turn. + const timeout = AbortSignal.timeout(CARRY_TIMEOUT_MS); + try { + await lifecycle.ensureUp(target, { lease, announce, signal: signal ? AbortSignal.any([signal, timeout]) : timeout }); + } catch (error) { + signal?.throwIfAborted(); + // Lifecycle labels all cancelled waits AbortError; preserve a timeout as a carry failure. + if (timeout.aborted) throw timeout.reason; + throw error; + } + signal?.throwIfAborted(); + const result = await exec.runExec(target, [target.container.name, "/bin/sh", "-c", script], { timeoutMs: CARRY_TIMEOUT_MS, + // The generic exec retry prepares without the carry lease. A vanished container is + // instead a recoverable carry failure. CLI exec itself is bounded, not abortable. + retry: false, }); + signal?.throwIfAborted(); + return result; } function cleanup(dir) { @@ -153,7 +170,7 @@ export function createContainerCarry({ exec, lifecycle, log = () => {} } = {}) { } } - async function copyIn(target, entries = []) { + async function copyIn(target, entries = [], opts = {}) { if (!target?.artifactDir) throw new Error("this target has no artifact dir — a carry has nowhere to stage"); const stagingDir = carryStagingDir(target, newCarryId()); try { @@ -171,7 +188,7 @@ export function createContainerCarry({ exec, lifecycle, log = () => {} } = {}) { } } if (!pairs.length) return { copied: 0 }; - const result = await runScript(target, buildCopyInScript(pairs)); + const result = await runScript(target, buildCopyInScript(pairs), opts); if (result.code !== 0) { throw new Error(`copying session state into ${target.container.name} failed: ${String(result.stderr || "").trim() || `exit ${result.code}`}`); } @@ -181,12 +198,12 @@ export function createContainerCarry({ exec, lifecycle, log = () => {} } = {}) { } } - async function copyOut(target, entries = []) { + async function copyOut(target, entries = [], opts = {}) { if (!target?.artifactDir) throw new Error("this target has no artifact dir — a carry has nowhere to stage"); const stagingDir = carryStagingDir(target, newCarryId()); try { mkdirSync(stagingDir, { recursive: true, mode: 0o700 }); - const result = await runScript(target, buildCopyOutScript(entries, stagingDir)); + const result = await runScript(target, buildCopyOutScript(entries, stagingDir), opts); if (result.code !== 0) { throw new Error(`copying session state out of ${target.container.name} failed: ${String(result.stderr || "").trim() || `exit ${result.code}`}`); } diff --git a/src/runtimes/container/index.js b/src/runtimes/container/index.js index 1a553523..0957d5ec 100644 --- a/src/runtimes/container/index.js +++ b/src/runtimes/container/index.js @@ -257,12 +257,12 @@ export const containerBackend = Object.freeze({ // The carry pair (contract.js OPTIONAL_METHODS): a thread's engine-native history follows it // when its channel changes runtime backend. Both halves go through the bind-mounted artifact // dir — the daemon cannot touch the HOME volume itself. See ./carry.js. - async copyIn(target, entries) { - return runtime().carry.copyIn(target, entries); + async copyIn(target, entries, opts = {}) { + return runtime().carry.copyIn(target, entries, opts); }, - async copyOut(target, entries) { - return runtime().carry.copyOut(target, entries); + async copyOut(target, entries, opts = {}) { + return runtime().carry.copyOut(target, entries, opts); }, // The relayed Codex login file (contract.js OPTIONAL_METHODS): staged through the artifact dir diff --git a/src/runtimes/contract.js b/src/runtimes/contract.js index a96672a6..cfcbac00 100644 --- a/src/runtimes/contract.js +++ b/src/runtimes/contract.js @@ -57,9 +57,10 @@ export const OPTIONAL_METHODS = Object.freeze([ // authenticate in this runtime?" gate. The local runtime has none: the daemon's own logins are // right there. "credentialError", - // copyIn(target, entries) / copyOut(target, entries) → Promise<{ copied: number }> — move + // copyIn(target, entries, opts) / copyOut(target, entries, opts) → Promise<{ copied: number }> — move // ENGINE STATE between the daemon's filesystem and this runtime's. `copyIn` writes into the - // runtime, `copyOut` reads out of it; both take CarryEntry[] (src/runtimes/copy.js) whose + // runtime, `copyOut` reads out of it. opts carries { lease, signal, announce } into preparation. + // Both take CarryEntry[] (src/runtimes/copy.js) whose // `from`/`to` are absolute on their own side and share the relative tail `rel`. They exist so a // thread's engine-native history follows it when its channel changes runtime backend — see // src/gateway/session-carry.js. A backend that declares neither simply cannot carry, and the @@ -202,11 +203,11 @@ export const HELPER_COMMANDS = Object.freeze([ * image's /opt/channelgate bundle). Unknown names throw. * * OPTIONAL (see OPTIONAL_METHODS): - * copyIn(target, entries) → Promise<{ copied }> — write engine state from the DAEMON's filesystem + * copyIn(target, entries, opts) → Promise<{ copied }> — write engine state from the DAEMON's filesystem * into this runtime. Host: a plain node:fs copy. Container: stage under the * bind-mounted artifact dir, then one exec inside that copies each entry * into place; the staging dir is removed either way. - * copyOut(target, entries) → Promise<{ copied }> — the same in reverse. Overwrite, never delete, and a + * copyOut(target, entries, opts) → Promise<{ copied }> — the same in reverse. Overwrite, never delete, and a * missing source is 0 copied rather than an error. * inspectState(target, request) → Promise<[{ path, mtimeMs, head }]> — the files inside this runtime that * match `request.globs` (carry-style: `*` inside a segment, never across diff --git a/test/container-carry.test.js b/test/container-carry.test.js index 1eda1461..5a86ddf1 100644 --- a/test/container-carry.test.js +++ b/test/container-carry.test.js @@ -224,3 +224,91 @@ test("boot: the idle reaper runs even with the gateway switch off, so a carry ca __resetContainerRuntime(); } }); + +test("sudo history carry rebuilds stale mounts without waiting on its own lease", async () => { + const { createFakeCli, inspectLine } = await import("./container-fake-cli.js"); + const { createContainerCli } = await import("../src/runtimes/container/cli.js"); + const { createContainerImage } = await import("../src/runtimes/container/image.js"); + const { createContainerLifecycle } = await import("../src/runtimes/container/lifecycle.js"); + const { createContainerReaper } = await import("../src/runtimes/container/reaper.js"); + const { currentInstallId } = await import("../src/runtimes/container/names.js"); + const { carrySession } = await import("../src/gateway/session-carry.js"); + const h = harness("carry-rebuild"); + const fake = createFakeCli({ routes: [ + { match: (a) => a[1] === "image" && a[2] === "inspect", result: { code: 0, stdout: "sha256:img|1.0.0" } }, + { match: (a) => a[1] === "inspect", result: { code: 0, stdout: inspectLine({ + name: h.target.container.name, install: currentInstallId(), fingerprint: "old", mountFingerprint: "old-mounts", + }) } }, + ] }); + const cli = createContainerCli({ exec: fake.exec }); + const reaper = createContainerReaper(); + const lifecycle = createContainerLifecycle({ cli, image: createContainerImage({ cli }), reaper, + sleep: async () => { throw new Error("carry must not wait for its own lease"); }, + }); + const carry = createContainerCarry({ lifecycle, exec: { async runExec() { return { code: 0 }; } } }); + const container = { ...h.target, runtime: { acquireLease: reaper.acquireLease, ...carry } }; + const result = await carrySession({ + engine: "claude", sessionId: SESSION, cwd: h.target.cwd, storedRuntime: { backend: "container" }, + target: { ...h.target, backend: "host", container: null }, resolveFor: () => container, log() {}, + }); + // Empty fake history is fine; the real lifecycle must still have recreated the old container. + assert.equal(result, null); + assert.equal(fake.find("rm").length, 1); + assert.ok(fake.find("run").some((args) => args.includes("--name") && args.includes(h.target.container.name))); + assert.equal(reaper.leaseCount(h.target.container.name), 0); +}); + +test("carry cancellation during preparation cleans staging and never executes the copy", async () => { + const h = harness("carry-cancel"); + const controller = new AbortController(); + const lease = { id: "carry:one" }; + const notices = []; + const carry = createContainerCarry({ + lifecycle: { async ensureUp(_target, opts) { + assert.equal(opts.lease, lease); + opts.announce("Waiting for another run"); + controller.abort(); + opts.signal.throwIfAborted(); + } }, + exec: { async runExec() { assert.fail("cancelled preparation must not copy"); } }, + }); + await assert.rejects(carry.copyOut(h.target, [], { lease, signal: controller.signal, announce: (s) => notices.push(s) }), { name: "AbortError" }); + assert.deepEqual(notices, ["Waiting for another run"]); + assert.deepEqual(walkFiles(path.join(h.target.artifactDir, "carry")), []); +}); + +test("carry preparation timeout remains a recoverable failure instead of a user Stop", async (t) => { + const timeout = new AbortController(); + timeout.abort(new DOMException("Carry preparation timed out", "TimeoutError")); + t.mock.method(AbortSignal, "timeout", () => timeout.signal); + const h = harness("carry-timeout"); + const carry = createContainerCarry({ + lifecycle: { async ensureUp() { + // The lifecycle deliberately uses AbortError for every cancelled container wait. + throw new DOMException("Run aborted while waiting", "AbortError"); + } }, + exec: { async runExec() { assert.fail("timed out preparation must not copy"); } }, + }); + await assert.rejects(carry.copyOut(h.target, []), { name: "TimeoutError" }); + assert.deepEqual(walkFiles(path.join(h.target.artifactDir, "carry")), []); +}); + +test("Stop during bounded copy prevents copied history being applied on the host", async () => { + const h = harness("carry-stop-copy"); + const controller = new AbortController(); + const destination = path.join(tempDir("carry-stopped-host-"), "session.jsonl"); + const carry = createContainerCarry({ + lifecycle: { async ensureUp() {} }, + exec: { async runExec(_target, _args, opts) { + assert.equal(opts.retry, false); + assert.equal(opts.timeoutMs, CARRY_TIMEOUT_MS); + controller.abort(); + return { code: 0 }; + } }, + }); + await assert.rejects(carry.copyOut(h.target, [{ rel: "session.jsonl", kind: "file", from: "/inside/session.jsonl", to: destination }], { + signal: controller.signal, + }), { name: "AbortError" }); + assert.equal(existsSync(destination), false); + assert.deepEqual(walkFiles(path.join(h.target.artifactDir, "carry")), []); +}); diff --git a/test/platform-googlechat.test.js b/test/platform-googlechat.test.js index 9192ec67..43e2629f 100644 --- a/test/platform-googlechat.test.js +++ b/test/platform-googlechat.test.js @@ -13,6 +13,7 @@ import { createPubSubPuller, createDedupe, isSubscriptionName } from "../src/pla import { parseChatEvent, normalizeMessage, mentionsBot, resolveThreadKey, createSeenThreads } from "../src/platforms/googlechat/events.js"; import { createGoogleChatConnector, toSpaceName } from "../src/platforms/googlechat/connector.js"; import { googleChatAdapter } from "../src/platforms/googlechat.js"; +import { createTransportManager } from "../src/platforms/manager.js"; import { startGoogleChat } from "../src/platforms/googlechat/transport.js"; ensureTestEnv(); @@ -445,3 +446,28 @@ test("stopping a puller never waits for a blocked handler or dispatches the rest await loop; assert.deepEqual(seen, [1]); }); + + +test("a live Google pull permission failure reaches manager health and stops inbox dispatch", async () => { + let failPull; + const response = new Promise((resolve) => { failPull = resolve; }); + let inboxStopped = false; + const manager = createTransportManager({ platform: "googlechat", log: { error() {} }, + start: (_config, { onFatal }) => startGoogleChat({ + subscription: "projects/cg-test/subscriptions/chat", onMessage() {}, onFatal, + capabilities: googleChatAdapter.capabilities, log: { info() {}, error() {} }, + deps: { auth: fakeAuth(), api: fakeChatApi(), connector: {}, + inbox: { start() {}, stop() { inboxStopped = true; } }, + fetchImpl: async () => response }, + }), + }); + assert.equal((await manager.connect({})).connected, true); + failPull(fail(403, JSON.stringify({ error: { message: "User not authorized" } }))); + await new Promise((resolve) => setImmediate(resolve)); + assert.equal(manager.snapshot().status, "error"); + assert.equal(manager.getConnector(), null); + assert.match(manager.snapshot().error, /Pub\/Sub Subscriber.*reconnect Google Chat/); + assert.equal(inboxStopped, true); + assert.equal(manager.getTransport().puller.running, false); + await manager.disconnect(); +}); diff --git a/test/platform-transport-manager.test.js b/test/platform-transport-manager.test.js new file mode 100644 index 00000000..74ce233b --- /dev/null +++ b/test/platform-transport-manager.test.js @@ -0,0 +1,48 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { createTransportManager } from "../src/platforms/manager.js"; + +const quiet = { error() {} }; + +test("a runtime transport failure removes the live connector and reconnect clears the error", async () => { + const callbacks = []; + let stopped = 0; + const manager = createTransportManager({ platform: "fixture", log: quiet, + start: async (_config, { onFatal }) => { + callbacks.push(onFatal); + return { connector: { id: callbacks.length }, stop: async () => { stopped++; } }; + }, + }); + assert.equal((await manager.connect({})).connected, true); + callbacks[0](new Error("permission denied")); + assert.equal(manager.snapshot().status, "error"); + assert.equal(manager.snapshot().connected, false); + assert.equal(manager.snapshot().error, "permission denied"); + assert.equal(manager.getConnector(), null); + assert.equal((await manager.connect({})).connected, true); + assert.equal(stopped, 1); + assert.equal(manager.snapshot().error, null); + callbacks[0](new Error("stale failure")); + assert.equal(manager.snapshot().connected, true); + assert.equal(manager.getConnector().id, 2); + await manager.disconnect(); + callbacks[1](new Error("late failure after disconnect")); + assert.equal(manager.snapshot().status, "disconnected"); + assert.equal(stopped, 2); +}); + +test("a failure during start cannot be overwritten by connected status", async () => { + let stopped = false; + const manager = createTransportManager({ platform: "fixture", log: quiet, + start: async (_config, { onFatal }) => { + onFatal(new Error("first pull denied")); + return { connector: {}, stop: async () => { stopped = true; } }; + }, + }); + const snapshot = await manager.connect({}); + assert.equal(snapshot.status, "error"); + assert.equal(snapshot.error, "first pull denied"); + assert.equal(manager.getConnector(), null); + await manager.disconnect(); + assert.equal(stopped, true); +}); diff --git a/test/session-carry.live.test.js b/test/session-carry.live.test.js new file mode 100644 index 00000000..392861a5 --- /dev/null +++ b/test/session-carry.live.test.js @@ -0,0 +1,75 @@ +// Fixture-only live proof: no engine prompt, real login or external message is sent. +// Real disposable containers prove both native history layouts survive a mount rebuild while +// carrySession holds its own lease. Opt in with CG_LIVE_CONTAINER=1. +import test from "node:test"; +import assert from "node:assert/strict"; +import { randomUUID } from "node:crypto"; +import { mkdirSync, writeFileSync, readFileSync, rmSync } from "node:fs"; +import path from "node:path"; +import { ensureTestEnv, tempDir } from "./helpers.js"; + +ensureTestEnv(); +const LIVE = process.env.CG_LIVE_CONTAINER === "1"; + +test("live: Claude and Codex fixture history roundtrips through a container rebuild with the carry lease held", { + skip: !LIVE && "opt-in: CG_LIVE_CONTAINER=1", timeout: 180_000, +}, async (t) => { + const { __setContainerRuntime, __resetContainerRuntime, stopContainerRuntime, containerBackend } = await import("../src/runtimes/container/index.js"); + const { resolveRuntime } = await import("../src/runtimes/resolve.js"); + const { carrySession } = await import("../src/gateway/session-carry.js"); + const { engineStateDir, engineSessionFiles } = await import("../src/engines/registry.js"); + const settings = { + cli: "auto", image: process.env.CG_LIVE_IMAGE || "channelgate/runtime:latest", + idleMinutes: 10, maxRunning: 8, pidsLimit: 1024, hasClaudeOauthToken: true, + }; + const runtime = __setContainerRuntime({ log() {} }); + const slug = `cg-carry-fixtures-${randomUUID().slice(0, 8)}`; + const target = resolveRuntime(slug, { platform: "slack", channelId: "C-CARRY-FIXTURE" }, { settings }); + const host = { ...target, backend: "host", container: null }; + let mayHaveContainer = false; + try { + const caps = await runtime.cli.probe(settings, { image: settings.image }); + if (!caps.ok) return t.skip(caps.reason); + const image = await runtime.image.inspect(caps, settings); + if (!image.present) return t.skip(image.reason); + const inspectId = async () => { + const result = await runtime.cli.runWith(caps, ["inspect", "--type", "container", "--format", "{{.Id}}", target.container.name]); + assert.equal(result.code, 0, result.stderr); + return String(result.stdout).trim(); + }; + mayHaveContainer = true; + for (const engine of ["claude", "codex"]) { + const sessionId = randomUUID(); + const rel = engine === "claude" + ? engineSessionFiles(engine, { cwd: target.cwd, sessionId })[0].rel + : `sessions/2026/10/02/rollout-fixture-${sessionId}.jsonl`; + const fixture = path.join(engineStateDir(engine, host), rel); + const body = `${JSON.stringify({ fixture: true, engine, sessionId })}\n`; + mkdirSync(path.dirname(fixture), { recursive: true }); + writeFileSync(fixture, body); + assert.deepEqual(await carrySession({ + engine, sessionId, cwd: target.cwd, storedRuntime: { backend: "host" }, target, + resolveFor: () => host, log() {}, + }), { direction: "host→container", files: 1 }); + const before = await inspectId(); + rmSync(fixture); + // Change a real bind source while retaining cwd and the persistent HOME volume. + target.cleanWorkDir = tempDir(`cg-carry-new-clean-${engine}-`); + assert.deepEqual(await carrySession({ + engine, sessionId, cwd: target.cwd, storedRuntime: { backend: "container" }, target: host, + resolveFor: () => target, log() {}, + }), { direction: "container→host", files: 1 }); + assert.notEqual(await inspectId(), before, `${engine}: stale mounts must recreate the container`); + assert.equal(readFileSync(fixture, "utf8"), body); + assert.equal(runtime.reaper.leaseCount(target.container.name), 0); + t.diagnostic(`${engine}: fixture roundtrip preserved bytes across a real container rebuild`); + } + } finally { + try { + if (mayHaveContainer) await containerBackend.destroy(target, { volumes: true, reason: "disposable session fixture cleanup" }); + } finally { + stopContainerRuntime(); + __resetContainerRuntime(); + } + } +}); diff --git a/test/session-carry.test.js b/test/session-carry.test.js index f36e342a..41bde669 100644 --- a/test/session-carry.test.js +++ b/test/session-carry.test.js @@ -300,3 +300,33 @@ test("carry: entries pair the engine's relative paths with both state dirs, shar // Identical state dirs mean there is no boundary to cross. assert.deepEqual(buildCarryEntries({ engine: "codex", cwd: "/w", sessionId: SESSION, fromDir: "/same", toDir: "/same" }), []); }); + +test("carry: cancellation propagates and releases its lease instead of healing", async () => { + const h = carryHarness("cancel"); + const controller = new AbortController(); + const announce = () => {}; + h.containerTarget.runtime.copyIn = async (_target, _entries, opts) => { + assert.equal(opts.signal, controller.signal); + assert.equal(opts.announce, announce); + assert.equal(typeof opts.lease.release, "function"); + controller.abort(); + throw controller.signal.reason; + }; + await assert.rejects(carrySession({ + engine: "claude", sessionId: SESSION, cwd: "/w/cancel", storedRuntime: "host", + target: h.containerTarget, signal: controller.signal, announce, log: h.log, + }), { name: "AbortError" }); + assert.equal(h.calls.leases[0].released, true); + assert.deepEqual(h.logs, []); +}); + +test("carry: a stopped turn never acquires a lease or starts copying", async () => { + const h = carryHarness("pre-cancel"); + const controller = new AbortController(); + controller.abort(); + await assert.rejects(carrySession({ + engine: "claude", sessionId: SESSION, cwd: "/w/cancel", target: h.containerTarget, + signal: controller.signal, log: h.log, + }), { name: "AbortError" }); + assert.equal(h.calls.leases.length, 0); +});