From 8863276bc14146fb25d753fffb312173d3353f77 Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 00:27:31 +0300 Subject: [PATCH 01/14] fix: stage files for Composio with the consumer keys this gateway stores stage_file_for_composio spent the stored Composio token on the REST upload API, which accepts only project API keys. The tokens a gateway in personal mode stores are consumer keys (ck_...), the hosted MCP's credential, and the REST endpoint rejects them under either header - so every stage failed with 'Invalid API key' and the model told the user their key was broken. The unit tests used a ck_ key against a REST stub, which is how this stayed green. With a consumer key the daemon now stages through the hosted MCP's own workbench: it opens its own MCP session on the same key, writes the file into the sandbox in 768 KB base64 chunks (one request above ~5 MB is rejected with 413), checks the md5 and mints the key with get_mount_file_s3_key. Verified live: that key uploads to Drive from a separate MCP session, which is the daemon/model split. Bytes still never pass through the model; the sandbox name is reduced to safe characters so a file name can never become Python source. Project keys keep the REST route. Consumer-key staging is capped at 25 MB. An upstream problem now answers 'Staging failed:'; only the confinement check answers 'Staging refused:'. The guide says a refusal is final and a failure is reported, not worked around by pushing the bytes through a tool. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- CHANGELOG.md | 11 ++ FEATURES.md | 12 ++ TEST-PLAN.md | 11 ++ src/gateway/composio-files.js | 182 +++++++++++++++++- .../gateway-usage/references/sharing-files.md | 13 ++ src/mcp/tools/file-sharing.js | 21 +- test/composio-files.test.js | 122 +++++++++++- test/file-sharing-tools.test.js | 77 +++++++- 8 files changed, 431 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bc49c00..cf9cd4f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,17 @@ product overview. > | Makeitfuture Sustainable Use License 1.1 | 2026-08-20 | never published | > | Makeitfuture Sustainable Use License 1.0 | 2026-08-06 | never published | +## Unreleased + +- Sending a generated file to Drive, Gmail and other Composio tools works again — or rather, works + for the first time on a gateway using personal Composio tokens. `stage_file_for_composio` sent the + stored token to Composio's REST upload, which only accepts project API keys; the tokens stored + here are consumer (MCP) keys, so every stage failed with "Invalid API key" and the model told the + user their key was broken. With a consumer key the gateway now stages through the hosted MCP's own + workbench, daemon-side and in chunks, so the file still never passes through the conversation + (limit 25 MB there, 100 MB with a project key). A real problem now reads `Staging failed:`; only + a path outside the conversation's folder reads `Staging refused:`. + ## 0.5.5 — 2026-09-24 - The runtime image no longer carries SSH host private keys. Installing `openssh-server` generated diff --git a/FEATURES.md b/FEATURES.md index 450413b..6958c05 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -1809,6 +1809,18 @@ A categorized catalog of what's shipped. Cross-linked to `TEST-PLAN.md` checks. the one spent, through the same precedence the MCP config uses; a named identity with no key is reported rather than silently replaced by the other one. The key never enters the container, never reaches the model, and never appears in an error message. Nothing is published. + The REST upload accepts only a Composio PROJECT API key. The keys this gateway stores in personal + mode are CONSUMER keys (`ck_…`, the hosted MCP's credential), which that endpoint rejects, so + with a consumer key the gateway stages through the hosted MCP's own workbench instead: the daemon + opens its own MCP session on the same key, writes the file into the sandbox in base64 chunks of + 768 KB (one request above ~5 MB is rejected), checks the md5 and mints the key with + `get_mount_file_s3_key` — a key the model's separate session can then use. Bytes still never + pass through the model. The sandbox path is a random directory plus a name reduced to safe + characters, so a file name can never become Python source. Consumer-key staging is capped at + 25 MB (project keys keep 100 MB). A path outside the working folder answers `Staging refused:`; + anything that goes wrong after that answers `Staging failed:`, and the guide tells the model a + refusal is final (no copying the file in to get around it) and a failure is reported, not + worked around by pushing the bytes through a tool itself. → TEST-PLAN: Composio file staging. - **Temporary public file links (`create_public_file_link`).** For destinations that ingest by URL rather than by body, and for a person who simply wants a link. One file from the channel's own diff --git a/TEST-PLAN.md b/TEST-PLAN.md index 989460f..c47080b 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -56,6 +56,17 @@ `agent` → the channel token), a named identity with no key is reported instead of falling back to the other, and a path escaping the channel folder is refused before any key is spent. Engine-independent: the tool runs daemon-side and no harness participates. +- [x] Consumer keys (`test/composio-files.test.js`, `test/file-sharing-tools.test.js`): a `ck_` key + never reaches the REST API; the file crosses as appended base64 chunks in one MCP session and + is md5-verified before `get_mount_file_s3_key`; an incomplete transfer, a sandbox error and an + oversize file (the 25 MB consumer cap) are errors without the key; a hostile file name never + reaches the Python source; a project (`ak_`) key keeps the REST route; an upstream failure + answers `Staging failed:` while only the confinement check answers `Staging refused:`. +- [x] Live proof of the route (2026-09-25, Xavier): a `ck_` key is rejected by the REST upload under + both `x-api-key` and `x-consumer-api-key` (401). Through the hosted MCP on the same key: a 1 MB + file stages in one call; 4 MB is rejected (413); a 3 MB file sent as four 768 KB appends + reassembles with a matching md5; and an `s3key` minted in one MCP session uploads successfully + to Drive from a SEPARATE session (the daemon/model split). Probe file deleted afterwards. - [ ] Live acceptance, Claude and Codex: in a fixture channel with a Composio connection, ask the agent to put a file it generated into Drive. Require it to call `stage_file_for_composio` (not a base64 relay, not a public link), then `GOOGLEDRIVE_UPLOAD_FILE` on the SAME identity, diff --git a/src/gateway/composio-files.js b/src/gateway/composio-files.js index 1df1715..257f697 100644 --- a/src/gateway/composio-files.js +++ b/src/gateway/composio-files.js @@ -66,6 +66,174 @@ function failure(step, status, body) { return new Error(`Composio ${step} failed (HTTP ${status})${detail ? `: ${detail}` : ""}`); } +// ── Consumer keys: staging through the Composio MCP workbench ───────────────────────────────── +// The tokens this gateway stores for `composio-user` / `composio-agent` are Composio CONSUMER keys +// (`ck_…`) — the credential of Composio's hosted MCP, sent as `x-consumer-api-key`. The REST upload +// endpoint above does not accept them under either header (401 "Invalid API key" / "No +// authentication provided"), so with a consumer key the three-step flow cannot work, and it never +// did: every personal-mode stage failed before this route existed. What a consumer key CAN do is +// run code in the MCP's own workbench, whose `get_mount_file_s3_key(path)` puts a file from the +// sandbox's /mnt/files into the same storage and returns the s3key GOOGLEDRIVE_UPLOAD_FILE & co. +// expect — and that key is usable from any later MCP session on the same identity, which is what +// lets the daemon stage here and the model use the result in its own session. +// +// It still runs in the daemon, for the same reason as the REST route: the key never reaches the +// model, and neither do the file's bytes. They cross as base64 inside the code the daemon sends, +// in chunks, because one request above ~5 MB of base64 is rejected (413); the sandbox keeps its +// files for the life of one MCP session, so the chunks append to one file and a final call checks +// the md5 before minting the key. That makes this route slower than the REST one, hence its own, +// lower size cap. +export const COMPOSIO_WORKBENCH_STAGE_MAX_BYTES = 25 * 1024 * 1024; +export const COMPOSIO_WORKBENCH_CHUNK_BYTES = 768 * 1024; +const WORKBENCH_TOOL = "COMPOSIO_REMOTE_WORKBENCH"; +const WORKBENCH_TIMEOUT_MS = 120_000; +const STAGE_MARKER = "CGSTAGE"; + +/** A Composio consumer (hosted-MCP) key, as opposed to a project API key the REST API accepts. */ +export function isConsumerKey(key) { + return /^ck_/.test(String(key || "")); +} + +// The name becomes part of Python source, so it is reduced to characters that cannot end the +// string literal or start a statement. The FileUploadable keeps the real name; only the sandbox +// path uses this one. +export function sandboxFileName(name) { + const cleaned = String(name || "").replace(/[^A-Za-z0-9._-]+/g, "_").replace(/^[._]+/, "").slice(0, 80); + return cleaned || "file"; +} + +function mcpClient({ url, key, fetchImpl }) { + let session = ""; + let nextId = 1; + async function rpc(method, params, { notify = false } = {}) { + const body = notify ? { jsonrpc: "2.0", method, params } : { jsonrpc: "2.0", id: nextId++, method, params }; + const response = await fetchImpl(url, { + method: "POST", + headers: { + "Content-Type": "application/json", + Accept: "application/json, text/event-stream", + "x-consumer-api-key": key, + ...(session ? { "mcp-session-id": session } : {}), + }, + body: JSON.stringify(body), + signal: AbortSignal.timeout(WORKBENCH_TIMEOUT_MS), + }); + const minted = response.headers?.get?.("mcp-session-id"); + if (minted) session = minted; + const raw = await response.text().catch(() => ""); + if (notify) return null; + if (!response.ok) throw failure(`workbench ${method}`, response.status, raw); + return parseRpc(raw); + } + return { + async open() { + const init = await rpc("initialize", { + protocolVersion: "2025-03-26", + capabilities: {}, + clientInfo: { name: "channelgate-stage", version: "1" }, + }); + if (init?.error) throw new Error(`Composio workbench refused the session: ${rpcMessage(init.error)}`); + await rpc("notifications/initialized", {}, { notify: true }); + }, + async run(code) { + const reply = await rpc("tools/call", { name: WORKBENCH_TOOL, arguments: { code_to_execute: code, thought: "Stage a channel file for a Composio tool." } }); + if (reply?.error) throw new Error(`Composio workbench call failed: ${rpcMessage(reply.error)}`); + return workbenchOutput(reply); + }, + }; +} + +// A JSON-RPC reply arrives either as plain JSON or as one or more server-sent events; the result +// is the last event that carries an id, a result or an error. +function parseRpc(raw) { + const text = String(raw || ""); + const events = text.split("\n").filter((line) => line.startsWith("data:")).map((line) => line.slice(5).trim()).filter(Boolean); + for (let i = events.length - 1; i >= 0; i--) { + try { + const message = JSON.parse(events[i]); + if (message.id !== undefined || message.result || message.error) return message; + } catch { + /* not JSON — keep looking */ + } + } + try { + return JSON.parse(text); + } catch { + throw new Error("Composio workbench returned an unreadable response"); + } +} + +function rpcMessage(error) { + return String(error?.message || JSON.stringify(error || {})).slice(0, 200); +} + +// The tool's text content is itself JSON (`{ data: { stdout, error }, successful }`). Return the +// sandbox's stdout, or throw with the sandbox's own error so a Python failure is not mistaken for +// an empty success. +function workbenchOutput(reply) { + const text = (reply?.result?.content || []).map((part) => part?.text || "").join("\n"); + let parsed = null; + try { + parsed = JSON.parse(text); + } catch { + return text; + } + const data = parsed?.data || {}; + const error = String(data.error || parsed?.error || "").trim(); + if (error || parsed?.successful === false) { + throw new Error(`Composio workbench error: ${(error || "the sandbox reported a failure").slice(0, 200)}`); + } + return String(data.stdout ?? data.results ?? text); +} + +/** + * Stage bytes through the Composio MCP workbench with a consumer key. `fetchImpl` is injectable so + * the tests drive the whole session; `chunkBytes` is injectable so they can force several chunks. + */ +export async function stageViaWorkbench({ + consumerKey, + mcpUrl, + bytes, + name, + md5, + fetchImpl = fetch, + chunkBytes = COMPOSIO_WORKBENCH_CHUNK_BYTES, +} = {}) { + if (!mcpUrl) throw new Error("no Composio MCP URL is configured for workbench staging"); + const client = mcpClient({ url: mcpUrl, key: consumerKey, fetchImpl }); + await client.open(); + // A fresh directory per stage, so two stages in one sandbox can never append to each other. + const target = `/mnt/files/channelgate-stage/${createHash("md5").update(`${md5}:${Date.now()}:${Math.random()}`).digest("hex").slice(0, 16)}/${sandboxFileName(name)}`; + const step = Math.max(1, Math.floor(chunkBytes)); + for (let offset = 0, first = true; offset < bytes.length || first; offset += step, first = false) { + const chunk = bytes.subarray(offset, offset + step).toString("base64"); + await client.run( + "import base64, os\n" + + `os.makedirs(os.path.dirname('${target}'), exist_ok=True)\n` + + `with open('${target}', '${first ? "wb" : "ab"}') as handle:\n` + + ` handle.write(base64.b64decode('${chunk}'))\n`, + ); + if (bytes.length === 0) break; + } + const output = await client.run( + "import hashlib, json\n" + + `data = open('${target}', 'rb').read()\n` + + `digest = hashlib.md5(data).hexdigest()\n` + + `key = get_mount_file_s3_key('${target}') if digest == '${md5}' else None\n` + + "key = key[0] if isinstance(key, (tuple, list)) else key\n" + + `print('${STAGE_MARKER}' + json.dumps({'md5': digest, 'bytes': len(data), 's3key': key}))\n`, + ); + const line = String(output).split("\n").find((entry) => entry.startsWith(STAGE_MARKER)); + if (!line) throw new Error("the Composio workbench did not report the staged file"); + const report = JSON.parse(line.slice(STAGE_MARKER.length)); + if (report.md5 !== md5 || report.bytes !== bytes.length) { + throw new Error("the file arrived in the Composio workbench incomplete; nothing was staged"); + } + const s3key = String(report.s3key || ""); + if (!s3key) throw new Error("the Composio workbench returned no storage key"); + return s3key; +} + /** * Stage one already-resolved absolute path into Composio storage. * @@ -85,7 +253,10 @@ export async function stageFileForComposio({ filename = "", mimetype = "", apiBase = "", + mcpUrl = "", maxBytes = COMPOSIO_STAGE_MAX_BYTES, + workbenchMaxBytes = COMPOSIO_WORKBENCH_STAGE_MAX_BYTES, + workbenchChunkBytes = COMPOSIO_WORKBENCH_CHUNK_BYTES, fetchImpl = fetch, } = {}) { const key = String(apiKey || ""); @@ -95,9 +266,11 @@ export async function stageFileForComposio({ const name = String(filename || path.basename(absolutePath || "")).trim(); if (!name) throw new Error("the file to stage needs a name"); + const consumer = isConsumerKey(key); + const limit = consumer ? Math.min(maxBytes, workbenchMaxBytes) : maxBytes; const info = await stat(absolutePath); if (!info.isFile()) throw new Error("only a regular file can be staged"); - if (info.size > maxBytes) throw new Error(`file is ${info.size} bytes; the staging limit is ${maxBytes}`); + if (info.size > limit) throw new Error(`file is ${info.size} bytes; the staging limit is ${limit}${consumer ? " with a Composio consumer key" : ""}`); // O_NOFOLLOW on the final open, for the same reason the download router uses it: the caller // proved this path at lookup time, and a symlink swapped in between then and now must not @@ -112,6 +285,12 @@ export async function stageFileForComposio({ const type = String(mimetype || "").trim() || guessMimeType(name); const md5 = createHash("md5").update(bytes).digest("hex"); + + if (consumer) { + const s3key = await stageViaWorkbench({ consumerKey: key, mcpUrl, bytes, name, md5, fetchImpl, chunkBytes: workbenchChunkBytes }); + return { file: { name, mimetype: type, s3key }, bytes: bytes.length, deduplicated: false, toolkit, tool, route: "workbench" }; + } + const base = String(apiBase || "").trim().replace(/\/+$/, "") || composioApiBase(); const requested = await fetchImpl(`${base}${COMPOSIO_UPLOAD_REQUEST_PATH}`, { @@ -141,5 +320,6 @@ export async function stageFileForComposio({ deduplicated: !presigned, toolkit, tool, + route: "rest", }; } diff --git a/src/gateway/gateway-usage/references/sharing-files.md b/src/gateway/gateway-usage/references/sharing-files.md index e4284e6..3df4a92 100644 --- a/src/gateway/gateway-usage/references/sharing-files.md +++ b/src/gateway/gateway-usage/references/sharing-files.md @@ -45,6 +45,19 @@ Two things that will bite you if you skip them: If the reply says Composio already held those bytes, that is a deduplication hit, not a failure — the `s3key` is good. +Two answers are final, and they mean different things: + +- **`Staging refused:`** — the path is not a file of this conversation (outside the working folder, + a symlink out of it, not a regular file). That path is not exported. Tell the user so. Do **not** + copy the file into the working folder, or read it and re-create it, to get around the refusal. +- **`Staging failed:`** — Composio could not take the file (network, service, size). Report the + failure in one line and stop. Do **not** push the file's contents through Composio's workbench, + code execution or any other tool yourself: that routes the bytes through the conversation, which + is exactly what staging exists to avoid. + +Size: up to 25 MB on a Composio consumer key (the kind this gateway normally stores), 100 MB on a +project API key. Past that, say the file is too large to stage rather than splitting it. + ## 3. A URL is the only way in → `create_public_file_link` Some APIs ingest by URL rather than by body (`GOOGLEDRIVE_UPLOAD_FROM_URL` and friends), and diff --git a/src/mcp/tools/file-sharing.js b/src/mcp/tools/file-sharing.js index 09aa9c7..d2c3ef5 100644 --- a/src/mcp/tools/file-sharing.js +++ b/src/mcp/tools/file-sharing.js @@ -18,7 +18,8 @@ import { z } from "zod"; import { effectiveWorkDir } from "../../gateway/folders.js"; import { openConfinedFile } from "../../gateway/confined-file.js"; -import { stageFileForComposio, COMPOSIO_STAGE_MAX_BYTES } from "../../gateway/composio-files.js"; +import { stageFileForComposio, COMPOSIO_STAGE_MAX_BYTES, COMPOSIO_WORKBENCH_STAGE_MAX_BYTES } from "../../gateway/composio-files.js"; +import { composioUrl } from "../../gateway/mcp-catalog.js"; import { createPublicFileLink, listPublicFileLinks, @@ -85,7 +86,8 @@ export function register(server, ctx) { "GMAIL_SEND_EMAIL attachments, SLACK_UPLOAD_FILE, …). Those tools accept NO path and NO base64 — this is " + "how a file you generated here reaches them. Pass the returned object straight through as the tool's file " + "argument. Nothing is made publicly reachable and the Composio key never leaves the gateway. " + - `Path is workspace-relative; the limit is ${formatBytes(COMPOSIO_STAGE_MAX_BYTES)}.`, + `Path is workspace-relative; the limit is ${formatBytes(COMPOSIO_WORKBENCH_STAGE_MAX_BYTES)} on a Composio ` + + `consumer (MCP) key and ${formatBytes(COMPOSIO_STAGE_MAX_BYTES)} on a project API key.`, inputSchema: { path: z.string().describe("File path relative to this channel's working folder."), tool: z.string().describe("The Composio tool slug the staged file is for, e.g. GOOGLEDRIVE_UPLOAD_FILE."), @@ -116,8 +118,15 @@ export function register(server, ctx) { // Confine and open BEFORE anything is sent anywhere. openConfinedFile proves the file is // still inside this channel's folder at open time, which is what stops a model-supplied // path (or a symlink swapped in behind it) reaching the operator home on a channel that - // mounts one. - const opened = await openConfinedFile(workspaceFor(slug, meta), relative); + // mounts one. Only THIS step is a refusal; everything after it is a delivery that either + // worked or failed, and saying "refused" for an upstream error misleads the model into + // blaming the user's key. + let opened; + try { + opened = await openConfinedFile(workspaceFor(slug, meta), relative); + } catch (error) { + return text(`Staging refused: ${clean(error)}`); + } handle = opened.handle; await handle.close(); handle = null; @@ -128,6 +137,7 @@ export function register(server, ctx) { toolSlug: tool, filename: filename || opened.name, mimetype, + mcpUrl: composioUrl(), }); await logEvent("composio_file_staged", { channel: channelId, @@ -138,6 +148,7 @@ export function register(server, ctx) { identity, tool: staged.tool, deduplicated: staged.deduplicated, + route: staged.route, }); return text( `Staged \`${opened.relative}\` (${formatBytes(staged.bytes)})${staged.deduplicated ? " — Composio already held these exact bytes" : ""} ` + @@ -146,7 +157,7 @@ export function register(server, ctx) { `Run the tool on \`composio-${identity}\` — the key that staged it is the only one that can see it.`, ); } catch (error) { - return text(`Staging refused: ${clean(error)}`); + return text(`Staging failed: ${clean(error)}`); } finally { await handle?.close().catch(() => {}); } diff --git a/test/composio-files.test.js b/test/composio-files.test.js index ead0845..b8b84de 100644 --- a/test/composio-files.test.js +++ b/test/composio-files.test.js @@ -9,7 +9,9 @@ import { mkdtemp, rm, writeFile } from "node:fs/promises"; import { COMPOSIO_UPLOAD_REQUEST_PATH, + isConsumerKey, normalizeSlug, + sandboxFileName, stageFileForComposio, toolkitFromToolSlug, } from "../src/gateway/composio-files.js"; @@ -33,7 +35,7 @@ test("staging requests a key, uploads the bytes and returns the FileUploadable", const calls = []; const staged = await stageFileForComposio({ - apiKey: "ck_secret", + apiKey: "ak_secret", absolutePath: file, toolSlug: "googledrive_upload_file", apiBase: "https://backend.example.test/", @@ -51,7 +53,7 @@ test("staging requests a key, uploads the bytes and returns the FileUploadable", const [request, upload] = calls; assert.equal(request.url, `https://backend.example.test${COMPOSIO_UPLOAD_REQUEST_PATH}`); assert.equal(request.init.method, "POST"); - assert.equal(request.init.headers["x-api-key"], "ck_secret"); + assert.equal(request.init.headers["x-api-key"], "ak_secret"); assert.deepEqual(JSON.parse(request.init.body), { // The tool slug is upper-cased and the toolkit derived from its leading segment, so a caller // never has to state the same thing twice. @@ -101,7 +103,7 @@ test("a failed upload request names the step and the status without leaking the const file = await scratchFile(t, "x.pdf", "x"); await assert.rejects( stageFileForComposio({ - apiKey: "ck_secret_value", + apiKey: "ak_secret_value", absolutePath: file, toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", fetchImpl: async () => ({ ok: false, status: 401, text: async () => "invalid api key" }), @@ -109,7 +111,7 @@ test("a failed upload request names the step and the status without leaking the (error) => { assert.match(error.message, /upload request failed \(HTTP 401\)/); assert.match(error.message, /invalid api key/); - assert.ok(!error.message.includes("ck_secret_value")); + assert.ok(!error.message.includes("ak_secret_value")); return true; }, ); @@ -178,3 +180,115 @@ test("unknown extensions fall back to octet-stream rather than guessing", () => assert.equal(guessMimeType("archive.unknownext"), "application/octet-stream"); assert.equal(guessMimeType("README"), "application/octet-stream"); }); + +// ── Consumer keys (`ck_…`, the hosted MCP's credential): staged through the MCP workbench ───── + +// A fake Composio MCP endpoint: answers initialize, swallows the notification, and executes the +// staging code's two shapes (append a base64 chunk / verify + mint a key) against an in-memory +// sandbox, the way the real workbench's stdout reports them. +function fakeWorkbench({ failWith = "", corrupt = false } = {}) { + const calls = []; + const files = new Map(); + const fetchImpl = async (url, init) => { + const rpc = JSON.parse(init.body); + calls.push({ url, key: init.headers["x-consumer-api-key"], rpc }); + const headers = new Map([["mcp-session-id", "wb-session"]]); + const reply = (payload) => ({ ok: true, status: 200, headers: { get: (h) => headers.get(h) || null }, text: async () => (payload === null ? "" : JSON.stringify(payload)) }); + if (rpc.id === undefined) return reply(null); + if (rpc.method !== "tools/call") return reply({ jsonrpc: "2.0", id: rpc.id, result: { capabilities: {} } }); + assert.equal(rpc.params.name, "COMPOSIO_REMOTE_WORKBENCH"); + const code = rpc.params.arguments.code_to_execute; + const target = code.match(/open\('([^']+)'/)[1]; + let stdout = ""; + const chunk = code.match(/b64decode\('([^']*)'\)/); + if (failWith) return reply({ jsonrpc: "2.0", id: rpc.id, result: { content: [{ type: "text", text: JSON.stringify({ data: { stdout: "", error: failWith }, successful: false }) }] } }); + if (chunk) { + const prior = code.includes("'wb'") ? Buffer.alloc(0) : files.get(target) || Buffer.alloc(0); + files.set(target, Buffer.concat([prior, Buffer.from(chunk[1], "base64")])); + } else { + let data = files.get(target) || Buffer.alloc(0); + if (corrupt) data = data.subarray(1); + const md5 = createHash("md5").update(data).digest("hex"); + stdout = `CGSTAGE${JSON.stringify({ md5, bytes: data.length, s3key: `wb/${path.basename(target)}` })}\n`; + } + return reply({ jsonrpc: "2.0", id: rpc.id, result: { content: [{ type: "text", text: JSON.stringify({ data: { stdout, error: "" }, successful: true }) }] } }); + }; + return { calls, files, fetchImpl }; +} + +test("a consumer key stages through the MCP workbench in chunks and never touches the REST API", async (t) => { + const contents = Buffer.from("0123456789abcdefghij"); // 20 bytes → 3 chunks of 8 + const file = await scratchFile(t, "report.pdf", contents); + const wb = fakeWorkbench(); + const staged = await stageFileForComposio({ + apiKey: "ck_consumer", + absolutePath: file, + toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", + mcpUrl: "https://mcp.example.test/mcp", + workbenchChunkBytes: 8, + fetchImpl: wb.fetchImpl, + }); + assert.equal(staged.route, "workbench"); + assert.deepEqual(staged.file, { name: "report.pdf", mimetype: guessMimeType("report.pdf"), s3key: "wb/report.pdf" }); + assert.ok(wb.calls.every((call) => call.url === "https://mcp.example.test/mcp" && call.key === "ck_consumer")); + const codes = wb.calls.filter((call) => call.rpc.method === "tools/call").map((call) => call.rpc.params.arguments.code_to_execute); + assert.equal(codes.length, 4, "three chunks plus the verification call"); + assert.match(codes[0], /'wb'/); + assert.ok(codes.slice(1, 3).every((code) => /'ab'/.test(code)), "later chunks append"); + assert.deepEqual([...wb.files.values()][0], contents, "the sandbox holds exactly the file's bytes"); + assert.match(codes[3], /get_mount_file_s3_key/); +}); + +test("the sandbox path cannot carry code: the file name is reduced to safe characters", async (t) => { + const file = await scratchFile(t, "plain.txt", "x"); + const wb = fakeWorkbench(); + const staged = await stageFileForComposio({ + apiKey: "ck_consumer", + absolutePath: file, + toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", + filename: "q'); import os; os.system('id') #.txt", + mcpUrl: "https://mcp.example.test/mcp", + fetchImpl: wb.fetchImpl, + }); + const code = wb.calls.find((call) => call.rpc.method === "tools/call").rpc.params.arguments.code_to_execute; + assert.ok(!code.includes("import os; os.system"), "the caller's name never reaches the Python source"); + assert.match(code, /channelgate-stage\/[0-9a-f]{16}\/q_import_os_os.system_id_.txt'/); + // The FileUploadable still carries the name the caller asked for. + assert.equal(staged.file.name, "q'); import os; os.system('id') #.txt"); +}); + +test("an incomplete transfer is refused rather than staged", async (t) => { + const file = await scratchFile(t, "data.csv", "a,b\n1,2\n"); + const wb = fakeWorkbench({ corrupt: true }); + await assert.rejects( + stageFileForComposio({ apiKey: "ck_consumer", absolutePath: file, toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", mcpUrl: "https://mcp.example.test/mcp", fetchImpl: wb.fetchImpl }), + /incomplete; nothing was staged/, + ); +}); + +test("a sandbox error surfaces as an error, without the key", async (t) => { + const file = await scratchFile(t, "data.csv", "a,b\n"); + const wb = fakeWorkbench({ failWith: "PermissionError: /mnt/files is read-only" }); + await assert.rejects( + stageFileForComposio({ apiKey: "ck_consumer", absolutePath: file, toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", mcpUrl: "https://mcp.example.test/mcp", fetchImpl: wb.fetchImpl }), + (error) => /PermissionError/.test(error.message) && !error.message.includes("ck_consumer"), + ); +}); + +test("a consumer key has its own, lower size cap, checked before any call", async (t) => { + const file = await scratchFile(t, "big.bin", Buffer.alloc(64)); + const wb = fakeWorkbench(); + await assert.rejects( + stageFileForComposio({ apiKey: "ck_consumer", absolutePath: file, toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", mcpUrl: "https://mcp.example.test/mcp", workbenchMaxBytes: 32, fetchImpl: wb.fetchImpl }), + /staging limit is 32 with a Composio consumer key/, + ); + assert.equal(wb.calls.length, 0); +}); + +test("consumer and project keys are told apart by prefix", () => { + assert.equal(isConsumerKey("ck_abc"), true); + assert.equal(isConsumerKey("ak_abc"), false); + assert.equal(isConsumerKey(""), false); + assert.equal(sandboxFileName("../../etc/passwd"), "etc_passwd"); + assert.equal(sandboxFileName(""), "file"); +}); diff --git a/test/file-sharing-tools.test.js b/test/file-sharing-tools.test.js index 7069765..80de380 100644 --- a/test/file-sharing-tools.test.js +++ b/test/file-sharing-tools.test.js @@ -15,6 +15,7 @@ import test from "node:test"; import assert from "node:assert/strict"; import http from "node:http"; +import { createHash } from "node:crypto"; import path from "node:path"; import { mkdirSync, writeFileSync } from "node:fs"; @@ -37,14 +38,40 @@ writeFileSync(path.join(WORKDIR, "work", "PROPOSAL.pdf"), "%PDF-1.7 proposal\n") mkdirSync(path.join(scratch, "elsewhere"), { recursive: true }); writeFileSync(path.join(scratch, "elsewhere", "secret.txt"), "not yours"); -// A stub standing in for Composio's REST API, so the tool's real staging path is exercised. +// A stub standing in for Composio, so the tool's real staging path is exercised on both routes: +// the REST upload API (project API keys) and the hosted MCP's workbench (consumer `ck_` keys — +// what this gateway actually stores). The workbench half reassembles the appended base64 chunks +// per sandbox path and answers the final md5 check the way the real sandbox prints it. const staged = []; +const sandbox = new Map(); +function workbenchReply(code) { + const target = (code.match(/open\('([^']+)'/) || [])[1] || ""; + const chunk = (code.match(/b64decode\('([^']*)'\)/) || [])[1]; + if (chunk !== undefined) { + const prior = code.includes("'wb'") ? Buffer.alloc(0) : (sandbox.get(target) || Buffer.alloc(0)); + sandbox.set(target, Buffer.concat([prior, Buffer.from(chunk, "base64")])); + return ""; + } + const data = sandbox.get(target) || Buffer.alloc(0); + const md5 = createHash("md5").update(data).digest("hex"); + return `CGSTAGE${JSON.stringify({ md5, bytes: data.length, s3key: `workbench/staged/${path.basename(target)}` })}\n`; +} const composio = http.createServer((req, res) => { let body = ""; req.on("data", (c) => (body += c)); req.on("end", () => { - staged.push({ url: req.url, method: req.method, key: req.headers["x-api-key"], body }); res.setHeader("Content-Type", "application/json"); + if (req.url.startsWith("/mcp")) { + const rpc = JSON.parse(body || "{}"); + staged.push({ url: req.url, method: rpc.method, key: req.headers["x-consumer-api-key"], body }); + res.setHeader("mcp-session-id", "stub-session"); + if (rpc.id === undefined) return res.end(""); + const result = rpc.method === "tools/call" + ? { content: [{ type: "text", text: JSON.stringify({ data: { stdout: workbenchReply(rpc.params.arguments.code_to_execute), error: "" }, successful: true }) }] } + : { protocolVersion: "2025-03-26", capabilities: {} }; + return res.end(JSON.stringify({ jsonrpc: "2.0", id: rpc.id, result })); + } + staged.push({ url: req.url, method: req.method, key: req.headers["x-api-key"], body }); res.end(JSON.stringify(req.url.includes("/files/upload/request") ? { key: "org/staged/PROPOSAL.pdf", new_presigned_url: `http://127.0.0.1:${composio.address().port}/put` } : {})); @@ -52,6 +79,7 @@ const composio = http.createServer((req, res) => { }); await new Promise((resolve) => composio.listen(0, "127.0.0.1", resolve)); process.env.COMPOSIO_API_BASE = `http://127.0.0.1:${composio.address().port}`; +process.env.COMPOSIO_MCP_URL = `http://127.0.0.1:${composio.address().port}/mcp`; test.after(() => composio.close()); function tools({ author = "U_AUTHOR", meta = {} } = {}) { @@ -66,7 +94,7 @@ function tools({ author = "U_AUTHOR", meta = {} } = {}) { return map; } -test("staging uses the named identity's key and returns the FileUploadable", async () => { +test("a consumer key stages through the Composio workbench on the named identity's key", async () => { staged.length = 0; await setUser("U_AUTHOR", { name: "Author", approved: true, composioToken: "ck_personal" }); const reply = await tools({ meta: { composioToken: "ck_channel" } }).get("stage_file_for_composio")({ @@ -76,13 +104,14 @@ test("staging uses the named identity's key and returns the FileUploadable", asy }); assert.match(reply, /Staged `work\/PROPOSAL\.pdf`/); - assert.match(reply, /"s3key": "org\/staged\/PROPOSAL\.pdf"/); + assert.match(reply, /"s3key": "workbench\/staged\/PROPOSAL\.pdf"/); assert.match(reply, /"mimetype": "application\/pdf"/); + // Consumer keys are the hosted MCP's credential: the REST upload API rejects them, so nothing + // may be sent there — every request is a JSON-RPC call to the MCP endpoint. + assert.ok(staged.length >= 3 && staged.every((call) => call.url.startsWith("/mcp")), "consumer keys never hit the REST API"); // "user" must spend the PERSONAL key, never the channel's — a file staged on one is invisible // to the other, and silently substituting identities is the bug this asserts against. - assert.equal(staged[0].key, "ck_personal"); - assert.equal(JSON.parse(staged[0].body).tool_slug, "GOOGLEDRIVE_UPLOAD_FILE"); - assert.equal(staged[1].method, "PUT"); + assert.ok(staged.every((call) => call.key === "ck_personal")); staged.length = 0; const agentReply = await tools({ meta: { composioToken: "ck_channel" } }).get("stage_file_for_composio")({ @@ -91,7 +120,39 @@ test("staging uses the named identity's key and returns the FileUploadable", asy identity: "agent", }); assert.match(agentReply, /composio-agent/); - assert.equal(staged[0].key, "ck_channel"); + assert.ok(staged.every((call) => call.key === "ck_channel")); +}); + +test("a project API key keeps the REST upload route", async () => { + staged.length = 0; + await setUser("U_PROJECT", { name: "Project", approved: true, composioToken: "ak_project" }); + const reply = await tools({ author: "U_PROJECT" }).get("stage_file_for_composio")({ + path: "work/PROPOSAL.pdf", + tool: "GOOGLEDRIVE_UPLOAD_FILE", + identity: "user", + }); + assert.match(reply, /"s3key": "org\/staged\/PROPOSAL\.pdf"/); + assert.equal(staged[0].key, "ak_project"); + assert.equal(JSON.parse(staged[0].body).tool_slug, "GOOGLEDRIVE_UPLOAD_FILE"); + assert.equal(staged[1].method, "PUT"); +}); + +test("an upstream failure is reported as a failure, not as a refusal", async () => { + staged.length = 0; + process.env.COMPOSIO_MCP_URL = "http://127.0.0.1:9/unreachable"; + try { + const reply = await tools({ meta: { composioToken: "ck_channel" } }).get("stage_file_for_composio")({ + path: "work/PROPOSAL.pdf", + tool: "GOOGLEDRIVE_UPLOAD_FILE", + identity: "agent", + }); + // "Refused" is reserved for the confinement check; the model reads it as a policy decision + // and, on a 401, told the user their own key was invalid. + assert.match(reply, /^Staging failed:/); + assert.ok(!reply.includes("ck_channel"), "the key never appears in the reply"); + } finally { + process.env.COMPOSIO_MCP_URL = `http://127.0.0.1:${composio.address().port}/mcp`; + } }); test("staging names the missing identity instead of falling back to the other one", async () => { From eb297e5407c0ebb0c6fedf0b83e6b22008218f17 Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 00:31:28 +0300 Subject: [PATCH 02/14] fix: make 'never recommend a blanket prune' a hard rule every run reads The rule against suggesting podman system prune, podman image prune -a, podman volume prune and docker system prune lived in the operating guide's routing row and administration.md. A live run asked 'what can we clean up?' never opened the guide, measured the disk with its own tools and recommended two of those prunes - on this host they delete every channel's home volume. The run on the other engine read the reference and warned against them. The one-line rule now sits in the managed CLAUDE.md block's hard rules, which every run of both engines loads. The block keeps its 4 KB budget (4,091 bytes): the header note was tightened to make room, and every command is named whole on one line. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- CHANGELOG.md | 9 +++++++++ FEATURES.md | 12 +++++++++--- TEST-PLAN.md | 6 ++++++ src/gateway/folders.js | 21 ++++++++++++++------- test/folders-generator-paths.test.js | 8 ++++++++ test/host-housekeeping-guide.test.js | 5 ++++- 6 files changed, 50 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bc49c00..5f32205 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,15 @@ product overview. > | Makeitfuture Sustainable Use License 1.1 | 2026-08-20 | never published | > | Makeitfuture Sustainable Use License 1.0 | 2026-08-06 | never published | +## Unreleased + +- The assistant no longer suggests a blanket container prune when asked about disk space. The + rule against `podman system prune`, `podman image prune -a`, `podman volume prune` and + `docker system prune` lived only in the operating guide, and a run that answered without opening + the guide recommended two of them — on this host they delete every channel's home. The rule now + sits among the few hard rules every run of every engine reads, pointing at the report-only + `npm run runtime:storage` instead. + ## 0.5.5 — 2026-09-24 - The runtime image no longer carries SSH host private keys. Installing `openssh-server` generated diff --git a/FEATURES.md b/FEATURES.md index 450413b..8962bea 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -2859,10 +2859,16 @@ are retired, bullet by bullet; everything else stands. rather than by reading the shared identity that holds other people's accounts (and the mirror: "your X" never touches `composio-user`); that `COMPOSIO_MANAGE_CONNECTIONS` initiates connections rather than listing them; and that only the gateway's `run_in_background` / - `run_agent_in_background` / `create_schedule` can report back after a turn ends — stated here + `run_agent_in_background` / `create_schedule` can report back after a turn ends; and that no run + runs or recommends a blanket container prune (`podman system prune`, `podman image prune -a`, + `podman volume prune`, `docker system prune`), which deletes every channel's home — disk cleanup + goes through the report-only `npm run runtime:storage` — stated here because a skill body is read only when the model opens it, and one engine reliably did not - (retest, 2026-09-06). They ride clean mode too, are engine-neutral, - and stay under 4 KB with the switches so the always-on prompt weight is read rather than skimmed. Editable three ways: the admin UI Instructions tab (edits the real + (retest, 2026-09-06; the prune rule after a 2026-09-25 run that skipped the skill and recommended + both prunes). They ride clean mode too, are engine-neutral, + and stay under 4 KB with the switches so the always-on prompt weight is read rather than skimmed + (4,091 bytes after the prune rule: the header note was tightened to make room, and the next + always-on rule has to trade space for it). Editable three ways: the admin UI Instructions tab (edits the real file; managed block shown read-only with a Settings link; hash-guarded against concurrent writes), by hand, or by asking the agent — the `update_channel_instructions` gateway MCP tool appends a rule in any mode (replace = admin-only). New sessions and `/clear` pick the file up diff --git a/TEST-PLAN.md b/TEST-PLAN.md index 989460f..d65ac7a 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -191,6 +191,12 @@ ## Host container-storage housekeeping guidance +- [x] `test/folders-generator-paths.test.js`: the managed CLAUDE.md block (read by every run of both + engines, skill opened or not) forbids running or recommending `podman system prune`, + `podman image prune -a`, `podman volume prune` and `docker system prune`, each named whole on + one line, and names `npm run runtime:storage` (report only); the gateway-owned block stays + under 4 KB. Live: OPS-DISK-01 (Airtable) on both engines — a 2026-09-25 Claude run never + opened the skill and recommended both prunes; the Codex run read administration.md and did not. - [x] `test/host-housekeeping-guide.test.js`: the materialized guide for every platform routes disk/stale-container/old-image questions to `references/administration.md` and carries the instruction to report and ask, removing only via `--apply` on an admin's word, and forbids diff --git a/src/gateway/folders.js b/src/gateway/folders.js index fded454..e7dd2b4 100644 --- a/src/gateway/folders.js +++ b/src/gateway/folders.js @@ -104,12 +104,9 @@ function stripBlock(content, start, end) { // survives /clear (a new session re-reads the file), and never touches anything below the marker. const GW_START = ""; const GW_END = ""; -const GW_NOTE = `> ⚙️ Gateway-managed block — do NOT edit between these markers; the gateway refreshes this -> section automatically (this conversation's switches, the gateway's hard rules, and the admin's -> global instructions). Everything BELOW the end marker is this channel's own standing instructions: -> it persists across sessions and is never overwritten. To add a durable channel rule when asked, -> use the gateway tool \`update_channel_instructions\` (or edit the file where file writes are -> allowed).`; +const GW_NOTE = `> ⚙️ Gateway-managed block: do NOT edit between these markers; the gateway rewrites it. Below the +> end marker are this channel's own standing instructions, never overwritten. To add one when +> asked, use \`update_channel_instructions\` (or edit this file if writable).`; // What each mode actually grants, in the agent's own terms — the label alone ("Bash") does not // tell a model what it may do. @@ -183,6 +180,13 @@ export function channelSwitchesNote(meta = {}) { // identities this turn actually received, next to the fresh-session memory catalog and the // caller's provenance note — per run, per author, so there is no shared file to race. Its text and // the predicate behind it live in src/gateway/mcp.js, beside the code that names those servers. +// The prune rule (OPS-DISK-01, 2026-09-25) is the same miss again. The routing row and +// administration.md already forbade suggesting a blanket prune after 0.5.3's L3 failure — but that +// text lives in the skill, and a Claude run asked "what can we clean up?" never opened it, measured +// the disk with its own tools and recommended `podman image prune -a` and `podman system prune`, +// while the Codex run, which read administration.md, warned against exactly those. On this host a +// blanket prune deletes every stopped channel's HOME volume (sessions, CLI logins, memory), so the +// one-line core belongs here. const HARD_RULES = `**Hard rules (not optional)** — they apply wherever the named tools exist; the reasoning and the tool shapes are in the \`gateway-usage\` skill: - Use \`ask_questions\` for clarification. @@ -215,7 +219,10 @@ tool shapes are in the \`gateway-usage\` skill: none of them, say so plainly instead of promising. A bounded "check every N minutes, K times" is one of these too: \`create_schedule\` (or \`run_agent_in_background\` for a self-contained watcher), never an in-turn sleep/poll loop, a \`Monitor\`-style wait, or a harness background task — even when - the loop would finish inside this turn.`; + the loop would finish inside this turn. +- **Never run or recommend a blanket prune** — \`podman system prune\`, \`podman image prune -a\`, + \`podman volume prune\`, \`docker system prune\` delete every channel's home. Cleanup: + \`npm run runtime:storage\` (report only).`; // Compose the managed block for a channel: the do-not-edit note, this conversation's switches, the // hard rules, and (outside clean mode) the admin's global instructions. Deliberately nothing about diff --git a/test/folders-generator-paths.test.js b/test/folders-generator-paths.test.js index a7646d2..8f7deeb 100644 --- a/test/folders-generator-paths.test.js +++ b/test/folders-generator-paths.test.js @@ -241,6 +241,14 @@ test("the managed block carries the hard rules a run must never get wrong", () = assert.match(block, /A bounded "check every N minutes, K times"/); assert.match(block, /never an in-turn sleep\/poll loop/); + // 5. No blanket container prune, ever — in the block every run loads, not only in the skill + // (OPS-DISK-01: a Claude run never opened the skill and recommended `podman system prune`). + assert.match(block, /Never run or recommend a blanket prune/); + for (const command of ["podman system prune", "podman image prune -a", "podman volume prune", "docker system prune"]) { + assert.ok(block.includes(`\`${command}\``), `${command} is named whole, on one line`); + } + assert.ok(block.includes("`npm run runtime:storage` (report only)")); + // Still the whole block, not a replacement for it: tonight's switches section survives. assert.match(block, /This conversation's switches/); } diff --git a/test/host-housekeeping-guide.test.js b/test/host-housekeeping-guide.test.js index 1642b9b..2680520 100644 --- a/test/host-housekeeping-guide.test.js +++ b/test/host-housekeeping-guide.test.js @@ -73,7 +73,10 @@ test("the guide sends the agent to the gateway's own report first, and to --appl // and then RECOMMENDED `podman system prune -a --volumes`, which deletes every stopped channel's // HOME volume. "Never delete on your own" did not stop it suggesting the command. The row it does // read must forbid the commands by name and point at the safe report. -test("the always-loaded routing row forbids suggesting a blanket prune and names the safe report", async () => { +// The routing row is only read when a run opens the skill. The same rule therefore also lives in +// the managed CLAUDE.md block's hard rules (test/folders-generator-paths.test.js), which every run +// of every engine loads — the row alone did not stop a run that never opened the skill (OPS-DISK-01). +test("the skill's routing row forbids suggesting a blanket prune and names the safe report", async () => { const cwd = tempDir("cg-housekeeping-prune-"); try { for (const platform of ["slack", "msteams", "googlechat"]) { From 19412818c504889d756fc2c2818e608bae31f5ed Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 00:36:41 +0300 Subject: [PATCH 03/14] fix: stage from the descriptor the confinement check proved Independent review of the consumer-key staging change found a race that predates it: the staging tool proved the file inside the channel folder with openConfinedFile (O_NOFOLLOW + /proc/self/fd), then closed that descriptor and let staging reopen the file by path, following symlinks. The container can write its own folder, so a symlink swapped in between proof and read would have sent any file the unsandboxed daemon can read. Staging now reads the proven descriptor, bounded to the cap from the same descriptor's size; direct callers get O_NOFOLLOW. No stage ever completed on a consumer-key gateway before this, so it was not usable. Also from the review: every error text is scrubbed of the key and of long base64 runs; the MCP reply must match the request id and carry no method (a server ping is not the answer), SSE data lines are joined; isError and successful:false fail; an overall 10-minute deadline bounds the session, which is ended with DELETE. Only a known project key (ak_) takes the REST route. The sandbox copy is deliberately kept: deleting it breaks the s3key (verified live). Tests cover the symlink swap, a file growing past the cap, session-id reuse, SSE with a leading ping, zero-length and exact-multiple files, every failure shape, key echo on both routes and the deadline; each of the four core claims was mutation-checked red. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- CHANGELOG.md | 11 +- FEATURES.md | 8 +- TEST-PLAN.md | 16 +- src/gateway/composio-files.js | 338 +++++++++++++++++++++------------- src/mcp/tools/file-sharing.js | 8 +- test/composio-files.test.js | 218 ++++++++++++++++------ 6 files changed, 408 insertions(+), 191 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cf9cd4f..581de1e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,8 +24,15 @@ product overview. here are consumer (MCP) keys, so every stage failed with "Invalid API key" and the model told the user their key was broken. With a consumer key the gateway now stages through the hosted MCP's own workbench, daemon-side and in chunks, so the file still never passes through the conversation - (limit 25 MB there, 100 MB with a project key). A real problem now reads `Staging failed:`; only - a path outside the conversation's folder reads `Staging refused:`. + (limit 25 MB there, 100 MB with a project key). A real problem now reads `Staging failed:`; + `Staging refused:` is kept for files that are not the conversation's own (outside its folder, a + symlink, not a regular file). +- Security: staging a file for Composio now reads the exact file whose location it checked. It + used to check the file, close it and open it again by name, and the conversation's container can + write its own folder — so in that gap the name could be swapped for a link to a file elsewhere on + the host, which the gateway would then have read and uploaded. Found by review during the + 2026-09-25 QA campaign. The tool first shipped in 0.5.3, and on a gateway using consumer keys (the + default) no stage could complete before this release, so the gap could not have been used there. ## 0.5.5 — 2026-09-24 diff --git a/FEATURES.md b/FEATURES.md index 6958c05..1dd8351 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -1817,7 +1817,13 @@ A categorized catalog of what's shipped. Cross-linked to `TEST-PLAN.md` checks. `get_mount_file_s3_key` — a key the model's separate session can then use. Bytes still never pass through the model. The sandbox path is a random directory plus a name reduced to safe characters, so a file name can never become Python source. Consumer-key staging is capped at - 25 MB (project keys keep 100 MB). A path outside the working folder answers `Staging refused:`; + 25 MB (project keys keep 100 MB), within an overall 10-minute deadline; the MCP session is ended + afterwards, but the sandbox copy is kept because the s3key IS that file's storage (deleting it + breaks the upload — verified live). The bytes are read from the SAME descriptor the confinement + check proved (O_NOFOLLOW + /proc/self/fd), bounded to the cap: the tool used to close that + descriptor and reopen the file by path, so a symlink swapped in by the container between proof + and read could have sent any file the daemon can read. Every error text is scrubbed of the key + and of long base64 runs before it reaches the model. A path outside the working folder answers `Staging refused:`; anything that goes wrong after that answers `Staging failed:`, and the guide tells the model a refusal is final (no copying the file in to get around it) and a failure is reported, not worked around by pushing the bytes through a tool itself. diff --git a/TEST-PLAN.md b/TEST-PLAN.md index c47080b..fc45fe2 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -56,12 +56,24 @@ `agent` → the channel token), a named identity with no key is reported instead of falling back to the other, and a path escaping the channel folder is refused before any key is spent. Engine-independent: the tool runs daemon-side and no harness participates. +- [x] Staging race (`test/composio-files.test.js`): with the proven descriptor open, the file's path + is replaced by a symlink to a file outside the folder — the staged bytes are still the proven + file's; a direct caller handing a symlinked path gets ELOOP (O_NOFOLLOW); a file that grows + past the cap after the size check is refused without being read whole. Mutation-checked: + reintroducing the reopen-by-path, dropping the session header, accepting a server ping as the + answer, or removing the key scrub each turns a test red. - [x] Consumer keys (`test/composio-files.test.js`, `test/file-sharing-tools.test.js`): a `ck_` key never reaches the REST API; the file crosses as appended base64 chunks in one MCP session and is md5-verified before `get_mount_file_s3_key`; an incomplete transfer, a sandbox error and an oversize file (the 25 MB consumer cap) are errors without the key; a hostile file name never - reaches the Python source; a project (`ak_`) key keeps the REST route; an upstream failure - answers `Staging failed:` while only the confinement check answers `Staging refused:`. + reaches the Python source; a project (`ak_`) key keeps the REST route and any other shape is + treated as a consumer key; every later request carries the minted MCP session id and the + session is ended with DELETE; SSE replies parse and a server ping (id + method) is never taken + for the answer; zero-length files and exact chunk multiples reassemble; a JSON-RPC error, an + HTTP error, `isError`, `successful:false` with and without text, and a hung server (overall + deadline) are all errors; an upstream body echoing the key is scrubbed on both routes; an + upstream failure answers `Staging failed:` while only the confinement check answers + `Staging refused:`. - [x] Live proof of the route (2026-09-25, Xavier): a `ck_` key is rejected by the REST upload under both `x-api-key` and `x-consumer-api-key` (401). Through the hosted MCP on the same key: a 1 MB file stages in one call; 4 MB is rejected (413); a 3 MB file sent as four 768 KB appends diff --git a/src/gateway/composio-files.js b/src/gateway/composio-files.js index 257f697..a0bf419 100644 --- a/src/gateway/composio-files.js +++ b/src/gateway/composio-files.js @@ -22,8 +22,9 @@ // precedence the MCP config uses, spent against Composio over TLS, and never returned to the // model, written to the workspace or logged. The only thing that crosses back is the opaque // `s3key`. -import { createHash } from "node:crypto"; -import { open, stat } from "node:fs/promises"; +import { createHash, randomBytes } from "node:crypto"; +import { constants } from "node:fs"; +import { open } from "node:fs/promises"; import path from "node:path"; import { guessMimeType } from "../util/mime.js"; @@ -82,16 +83,30 @@ function failure(step, status, body) { // in chunks, because one request above ~5 MB of base64 is rejected (413); the sandbox keeps its // files for the life of one MCP session, so the chunks append to one file and a final call checks // the md5 before minting the key. That makes this route slower than the REST one, hence its own, -// lower size cap. +// lower size cap and an overall deadline. +// +// The sandbox copy is deliberately NOT deleted afterwards: the s3key IS that mounted file's +// storage, and removing the file makes the upload fail with "the file does not exist in storage" +// (verified live). It lives in the same identity's own Composio sandbox as any workbench file. The +// MCP session itself is ended (HTTP DELETE) — that does not affect the key (also verified). export const COMPOSIO_WORKBENCH_STAGE_MAX_BYTES = 25 * 1024 * 1024; export const COMPOSIO_WORKBENCH_CHUNK_BYTES = 768 * 1024; +export const COMPOSIO_WORKBENCH_DEADLINE_MS = 10 * 60_000; const WORKBENCH_TOOL = "COMPOSIO_REMOTE_WORKBENCH"; -const WORKBENCH_TIMEOUT_MS = 120_000; +const WORKBENCH_CALL_TIMEOUT_MS = 120_000; const STAGE_MARKER = "CGSTAGE"; -/** A Composio consumer (hosted-MCP) key, as opposed to a project API key the REST API accepts. */ +/** + * A Composio PROJECT API key — the only kind the REST upload accepts. Anything else takes the + * workbench route: the tokens a gateway stores are consumer keys, and src/util/redact.js notes + * Composio tokens carry no guaranteed prefix, so the unknown case goes where stored tokens work. + */ +export function isProjectApiKey(key) { + return /^ak_/.test(String(key || "")); +} + export function isConsumerKey(key) { - return /^ck_/.test(String(key || "")); + return Boolean(String(key || "")) && !isProjectApiKey(key); } // The name becomes part of Python source, so it is reduced to characters that cannot end the @@ -102,28 +117,46 @@ export function sandboxFileName(name) { return cleaned || "file"; } -function mcpClient({ url, key, fetchImpl }) { +// Every message that can reach the model passes through here. The daemon never writes the key into +// an error itself, but an upstream body can echo it back; and a sandbox traceback can quote a line +// of the base64 the daemon sent. Neither belongs in a reply. +export function scrubStagingText(text, key = "") { + let out = String(text || ""); + if (key && String(key).length >= 6) out = out.split(String(key)).join("[redacted]"); + return out.replace(/[A-Za-z0-9+/=]{48,}/g, "[data]"); +} + +function mcpClient({ url, key, fetchImpl, deadline }) { let session = ""; let nextId = 1; - async function rpc(method, params, { notify = false } = {}) { - const body = notify ? { jsonrpc: "2.0", method, params } : { jsonrpc: "2.0", id: nextId++, method, params }; - const response = await fetchImpl(url, { - method: "POST", - headers: { - "Content-Type": "application/json", - Accept: "application/json, text/event-stream", - "x-consumer-api-key": key, - ...(session ? { "mcp-session-id": session } : {}), - }, - body: JSON.stringify(body), - signal: AbortSignal.timeout(WORKBENCH_TIMEOUT_MS), - }); - const minted = response.headers?.get?.("mcp-session-id"); - if (minted) session = minted; - const raw = await response.text().catch(() => ""); - if (notify) return null; + const signal = () => AbortSignal.any([AbortSignal.timeout(WORKBENCH_CALL_TIMEOUT_MS), deadline]); + async function post(body) { + try { + const response = await fetchImpl(url, { + method: "POST", + headers: { + "Content-Type": "application/json", + Accept: "application/json, text/event-stream", + "x-consumer-api-key": key, + ...(session ? { "mcp-session-id": session } : {}), + }, + body: JSON.stringify(body), + signal: signal(), + }); + const minted = response.headers?.get?.("mcp-session-id"); + if (minted) session = minted; + const raw = await response.text(); + return { response, raw }; + } catch (error) { + if (error?.name === "TimeoutError" || error?.name === "AbortError") throw new Error("the Composio workbench timed out"); + throw error; + } + } + async function rpc(method, params) { + const id = nextId++; + const { response, raw } = await post({ jsonrpc: "2.0", id, method, params }); if (!response.ok) throw failure(`workbench ${method}`, response.status, raw); - return parseRpc(raw); + return parseRpc(raw, id); } return { async open() { @@ -133,34 +166,48 @@ function mcpClient({ url, key, fetchImpl }) { clientInfo: { name: "channelgate-stage", version: "1" }, }); if (init?.error) throw new Error(`Composio workbench refused the session: ${rpcMessage(init.error)}`); - await rpc("notifications/initialized", {}, { notify: true }); + await post({ jsonrpc: "2.0", method: "notifications/initialized", params: {} }); }, async run(code) { const reply = await rpc("tools/call", { name: WORKBENCH_TOOL, arguments: { code_to_execute: code, thought: "Stage a channel file for a Composio tool." } }); if (reply?.error) throw new Error(`Composio workbench call failed: ${rpcMessage(reply.error)}`); return workbenchOutput(reply); }, + async close() { + if (!session) return; + await fetchImpl(url, { method: "DELETE", headers: { "x-consumer-api-key": key, "mcp-session-id": session }, signal: AbortSignal.timeout(10_000) }).catch(() => {}); + }, }; } -// A JSON-RPC reply arrives either as plain JSON or as one or more server-sent events; the result -// is the last event that carries an id, a result or an error. -function parseRpc(raw) { +// A JSON-RPC reply arrives either as plain JSON or as server-sent events. Only the message that +// answers THIS request counts: same id, and no `method` (a server-to-client request such as a +// ping also carries an id). An event's data may span several `data:` lines. +function parseRpc(raw, id) { const text = String(raw || ""); - const events = text.split("\n").filter((line) => line.startsWith("data:")).map((line) => line.slice(5).trim()).filter(Boolean); - for (let i = events.length - 1; i >= 0; i--) { + const candidates = []; + if (/^\s*[[{]/.test(text)) { try { - const message = JSON.parse(events[i]); - if (message.id !== undefined || message.result || message.error) return message; + const parsed = JSON.parse(text); + candidates.push(...(Array.isArray(parsed) ? parsed : [parsed])); } catch { - /* not JSON — keep looking */ + /* fall through to SSE */ } } - try { - return JSON.parse(text); - } catch { - throw new Error("Composio workbench returned an unreadable response"); + if (!candidates.length) { + for (const event of text.split(/\r?\n\r?\n/)) { + const data = event.split(/\r?\n/).filter((line) => line.startsWith("data:")).map((line) => line.slice(5).replace(/^ /, "")).join("\n"); + if (!data) continue; + try { + candidates.push(JSON.parse(data)); + } catch { + /* not JSON — skip */ + } + } } + const match = candidates.find((message) => message && message.id === id && !message.method); + if (!match) throw new Error("the Composio workbench returned no answer to the request"); + return match; } function rpcMessage(error) { @@ -169,9 +216,10 @@ function rpcMessage(error) { // The tool's text content is itself JSON (`{ data: { stdout, error }, successful }`). Return the // sandbox's stdout, or throw with the sandbox's own error so a Python failure is not mistaken for -// an empty success. +// an empty success. An MCP-level tool error (`isError`) is a failure whatever its text. function workbenchOutput(reply) { const text = (reply?.result?.content || []).map((part) => part?.text || "").join("\n"); + if (reply?.result?.isError) throw new Error(`Composio workbench error: ${text.slice(0, 200) || "the tool reported a failure"}`); let parsed = null; try { parsed = JSON.parse(text); @@ -198,56 +246,85 @@ export async function stageViaWorkbench({ md5, fetchImpl = fetch, chunkBytes = COMPOSIO_WORKBENCH_CHUNK_BYTES, + deadlineMs = COMPOSIO_WORKBENCH_DEADLINE_MS, } = {}) { if (!mcpUrl) throw new Error("no Composio MCP URL is configured for workbench staging"); - const client = mcpClient({ url: mcpUrl, key: consumerKey, fetchImpl }); - await client.open(); - // A fresh directory per stage, so two stages in one sandbox can never append to each other. - const target = `/mnt/files/channelgate-stage/${createHash("md5").update(`${md5}:${Date.now()}:${Math.random()}`).digest("hex").slice(0, 16)}/${sandboxFileName(name)}`; - const step = Math.max(1, Math.floor(chunkBytes)); - for (let offset = 0, first = true; offset < bytes.length || first; offset += step, first = false) { - const chunk = bytes.subarray(offset, offset + step).toString("base64"); - await client.run( - "import base64, os\n" + - `os.makedirs(os.path.dirname('${target}'), exist_ok=True)\n` + - `with open('${target}', '${first ? "wb" : "ab"}') as handle:\n` + - ` handle.write(base64.b64decode('${chunk}'))\n`, + const client = mcpClient({ url: mcpUrl, key: consumerKey, fetchImpl, deadline: AbortSignal.timeout(deadlineMs) }); + try { + await client.open(); + // A fresh directory per stage, so two stages in one sandbox can never append to each other. + const target = `/mnt/files/channelgate-stage/${randomBytes(8).toString("hex")}/${sandboxFileName(name)}`; + const step = Math.max(1, Math.floor(chunkBytes)); + for (let offset = 0, first = true; offset < bytes.length || first; offset += step, first = false) { + const chunk = bytes.subarray(offset, offset + step).toString("base64"); + await client.run( + "import base64, os\n" + + `os.makedirs(os.path.dirname('${target}'), exist_ok=True)\n` + + `with open('${target}', '${first ? "wb" : "ab"}') as handle:\n` + + ` handle.write(base64.b64decode('${chunk}'))\n`, + ); + if (bytes.length === 0) break; + } + const output = await client.run( + "import hashlib, json\n" + + `data = open('${target}', 'rb').read()\n` + + `digest = hashlib.md5(data).hexdigest()\n` + + `key = get_mount_file_s3_key('${target}') if digest == '${md5}' else None\n` + + "key = key[0] if isinstance(key, (tuple, list)) else key\n" + + `print('${STAGE_MARKER}' + json.dumps({'md5': digest, 'bytes': len(data), 's3key': key}))\n`, ); - if (bytes.length === 0) break; + const line = String(output).split("\n").find((entry) => entry.startsWith(STAGE_MARKER)); + if (!line) throw new Error("the Composio workbench did not report the staged file"); + const report = JSON.parse(line.slice(STAGE_MARKER.length)); + if (report.md5 !== md5 || report.bytes !== bytes.length) { + throw new Error("the file arrived in the Composio workbench incomplete; nothing was staged"); + } + const s3key = String(report.s3key || ""); + if (!s3key) throw new Error("the Composio workbench returned no storage key"); + return s3key; + } finally { + await client.close(); } - const output = await client.run( - "import hashlib, json\n" + - `data = open('${target}', 'rb').read()\n` + - `digest = hashlib.md5(data).hexdigest()\n` + - `key = get_mount_file_s3_key('${target}') if digest == '${md5}' else None\n` + - "key = key[0] if isinstance(key, (tuple, list)) else key\n" + - `print('${STAGE_MARKER}' + json.dumps({'md5': digest, 'bytes': len(data), 's3key': key}))\n`, - ); - const line = String(output).split("\n").find((entry) => entry.startsWith(STAGE_MARKER)); - if (!line) throw new Error("the Composio workbench did not report the staged file"); - const report = JSON.parse(line.slice(STAGE_MARKER.length)); - if (report.md5 !== md5 || report.bytes !== bytes.length) { - throw new Error("the file arrived in the Composio workbench incomplete; nothing was staged"); +} + +// Read at most `limit` bytes from an already-open descriptor. The size is taken from the SAME +// descriptor (never a second stat of a path), and the read stops one byte past the limit, so a file +// that grows after the check can neither exceed the cap nor be swapped for another file. +async function readBounded(handle, limit, consumer) { + const tooLarge = (size) => new Error(`file is ${size} bytes; the staging limit is ${limit}${consumer ? " with a Composio consumer key" : ""}`); + const info = await handle.stat(); + if (!info.isFile()) throw new Error("only a regular file can be staged"); + if (info.size > limit) throw tooLarge(info.size); + const parts = []; + let total = 0; + const buffer = Buffer.allocUnsafe(64 * 1024); + for (;;) { + const { bytesRead } = await handle.read(buffer, 0, buffer.length, total); + if (!bytesRead) break; + total += bytesRead; + if (total > limit) throw tooLarge(`more than ${limit}`); + parts.push(Buffer.from(buffer.subarray(0, bytesRead))); } - const s3key = String(report.s3key || ""); - if (!s3key) throw new Error("the Composio workbench returned no storage key"); - return s3key; + return Buffer.concat(parts, total); } /** - * Stage one already-resolved absolute path into Composio storage. + * Stage one file into Composio storage. * - * `absolutePath` must already have been confined to the channel's working folder by the caller — - * this module does no path authorization of its own and must never be handed a model-supplied - * path directly. `fetchImpl` is injectable so the tests can drive the whole three-step flow - * without a network. + * Pass `handle` — the descriptor `openConfinedFile` proved to be inside the channel's working + * folder — and the bytes are read from THAT descriptor. Reopening the file by path (as this module + * once did) let a symlink swapped in between the proof and the read redirect the daemon, which runs + * unsandboxed, to any file the operator can read. `absolutePath` remains for direct callers and is + * opened with O_NOFOLLOW; this module does no path authorization of its own and must never be + * handed a model-supplied path. `fetchImpl` is injectable so the tests drive the whole flow. * * Returns the exact `FileUploadable` the Composio tool expects, plus `deduplicated` so the caller - * can say whether bytes actually moved. + * can say whether bytes actually moved. Every error message is scrubbed of the key. */ export async function stageFileForComposio({ apiKey, - absolutePath, + handle = null, + absolutePath = "", toolSlug, toolkitSlug = "", filename = "", @@ -257,69 +334,74 @@ export async function stageFileForComposio({ maxBytes = COMPOSIO_STAGE_MAX_BYTES, workbenchMaxBytes = COMPOSIO_WORKBENCH_STAGE_MAX_BYTES, workbenchChunkBytes = COMPOSIO_WORKBENCH_CHUNK_BYTES, + workbenchDeadlineMs = COMPOSIO_WORKBENCH_DEADLINE_MS, fetchImpl = fetch, } = {}) { const key = String(apiKey || ""); - if (!key) throw new Error("no Composio API key resolved for this identity"); - const tool = normalizeSlug(toolSlug, "tool"); - const toolkit = toolkitSlug ? normalizeSlug(toolkitSlug, "toolkit") : toolkitFromToolSlug(tool); - const name = String(filename || path.basename(absolutePath || "")).trim(); - if (!name) throw new Error("the file to stage needs a name"); - - const consumer = isConsumerKey(key); - const limit = consumer ? Math.min(maxBytes, workbenchMaxBytes) : maxBytes; - const info = await stat(absolutePath); - if (!info.isFile()) throw new Error("only a regular file can be staged"); - if (info.size > limit) throw new Error(`file is ${info.size} bytes; the staging limit is ${limit}${consumer ? " with a Composio consumer key" : ""}`); - - // O_NOFOLLOW on the final open, for the same reason the download router uses it: the caller - // proved this path at lookup time, and a symlink swapped in between then and now must not - // redirect the read. - const handle = await open(absolutePath, "r"); - let bytes; try { - bytes = await handle.readFile(); - } finally { - await handle.close().catch(() => {}); + return await stage(); + } catch (error) { + throw new Error(scrubStagingText(error?.message || String(error), key)); } - const type = String(mimetype || "").trim() || guessMimeType(name); - const md5 = createHash("md5").update(bytes).digest("hex"); + async function stage() { + if (!key) throw new Error("no Composio API key resolved for this identity"); + const tool = normalizeSlug(toolSlug, "tool"); + const toolkit = toolkitSlug ? normalizeSlug(toolkitSlug, "toolkit") : toolkitFromToolSlug(tool); + const name = String(filename || path.basename(absolutePath || "")).trim(); + if (!name) throw new Error("the file to stage needs a name"); - if (consumer) { - const s3key = await stageViaWorkbench({ consumerKey: key, mcpUrl, bytes, name, md5, fetchImpl, chunkBytes: workbenchChunkBytes }); - return { file: { name, mimetype: type, s3key }, bytes: bytes.length, deduplicated: false, toolkit, tool, route: "workbench" }; - } + const consumer = isConsumerKey(key); + const limit = consumer ? Math.min(maxBytes, workbenchMaxBytes) : maxBytes; + let bytes; + if (handle) { + bytes = await readBounded(handle, limit, consumer); + } else { + const own = await open(absolutePath, constants.O_RDONLY | constants.O_NOFOLLOW); + try { + bytes = await readBounded(own, limit, consumer); + } finally { + await own.close().catch(() => {}); + } + } - const base = String(apiBase || "").trim().replace(/\/+$/, "") || composioApiBase(); + const type = String(mimetype || "").trim() || guessMimeType(name); + const md5 = createHash("md5").update(bytes).digest("hex"); - const requested = await fetchImpl(`${base}${COMPOSIO_UPLOAD_REQUEST_PATH}`, { - method: "POST", - headers: { "Content-Type": "application/json", "x-api-key": key }, - body: JSON.stringify({ toolkit_slug: toolkit, tool_slug: tool, filename: name, mimetype: type, md5 }), - }); - if (!requested.ok) throw failure("upload request", requested.status, await requested.text().catch(() => "")); - const grant = (await requested.json().catch(() => ({}))) || {}; - const s3key = String(grant.key || grant.s3key || ""); - if (!s3key) throw new Error("Composio returned no storage key for the upload request"); + if (consumer) { + const s3key = await stageViaWorkbench({ consumerKey: key, mcpUrl, bytes, name, md5, fetchImpl, chunkBytes: workbenchChunkBytes, deadlineMs: workbenchDeadlineMs }); + return { file: { name, mimetype: type, s3key }, bytes: bytes.length, deduplicated: false, toolkit, tool, route: "workbench" }; + } - // Dedup hit: Composio already holds these exact bytes and mints no presigned URL for them. - const presigned = String(grant.new_presigned_url || grant.presigned_url || ""); - if (presigned) { - const put = await fetchImpl(presigned, { - method: "PUT", - headers: { "Content-Type": type, "Content-Length": String(bytes.length) }, - body: bytes, + const base = String(apiBase || "").trim().replace(/\/+$/, "") || composioApiBase(); + const requested = await fetchImpl(`${base}${COMPOSIO_UPLOAD_REQUEST_PATH}`, { + method: "POST", + headers: { "Content-Type": "application/json", "x-api-key": key }, + body: JSON.stringify({ toolkit_slug: toolkit, tool_slug: tool, filename: name, mimetype: type, md5 }), }); - if (!put.ok) throw failure("storage upload", put.status, await put.text().catch(() => "")); - } + if (!requested.ok) throw failure("upload request", requested.status, await requested.text().catch(() => "")); + const grant = (await requested.json().catch(() => ({}))) || {}; + const s3key = String(grant.key || grant.s3key || ""); + if (!s3key) throw new Error("Composio returned no storage key for the upload request"); - return { - file: { name, mimetype: type, s3key }, - bytes: bytes.length, - deduplicated: !presigned, - toolkit, - tool, - route: "rest", - }; + // Dedup hit: Composio already holds these exact bytes and mints no presigned URL for them. + const presigned = String(grant.new_presigned_url || grant.presigned_url || ""); + if (presigned) { + const put = await fetchImpl(presigned, { + method: "PUT", + headers: { "Content-Type": type, "Content-Length": String(bytes.length) }, + body: bytes, + }); + if (!put.ok) throw failure("storage upload", put.status, await put.text().catch(() => "")); + } + + return { + file: { name, mimetype: type, s3key }, + bytes: bytes.length, + deduplicated: !presigned, + toolkit, + tool, + route: "rest", + }; + } } diff --git a/src/mcp/tools/file-sharing.js b/src/mcp/tools/file-sharing.js index d2c3ef5..d94699f 100644 --- a/src/mcp/tools/file-sharing.js +++ b/src/mcp/tools/file-sharing.js @@ -127,13 +127,13 @@ export function register(server, ctx) { } catch (error) { return text(`Staging refused: ${clean(error)}`); } + // Stage from the descriptor that was just PROVEN to be inside the folder — never reopen the + // file by path. The container can write this folder, so a path reopened after the proof + // could by then be a symlink into the operator's home, read by the unsandboxed daemon. handle = opened.handle; - await handle.close(); - handle = null; - const staged = await stageFileForComposio({ apiKey: key, - absolutePath: opened.realPath, + handle, toolSlug: tool, filename: filename || opened.name, mimetype, diff --git a/test/composio-files.test.js b/test/composio-files.test.js index b8b84de..c436c98 100644 --- a/test/composio-files.test.js +++ b/test/composio-files.test.js @@ -5,13 +5,16 @@ import assert from "node:assert/strict"; import os from "node:os"; import path from "node:path"; import { createHash } from "node:crypto"; -import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { constants } from "node:fs"; +import { mkdtemp, open, rm, symlink, unlink, writeFile } from "node:fs/promises"; import { COMPOSIO_UPLOAD_REQUEST_PATH, isConsumerKey, + isProjectApiKey, normalizeSlug, sandboxFileName, + scrubStagingText, stageFileForComposio, toolkitFromToolSlug, } from "../src/gateway/composio-files.js"; @@ -81,7 +84,7 @@ test("a deduplication hit returns the key without a second request", async (t) = let calls = 0; const staged = await stageFileForComposio({ - apiKey: "ck", + apiKey: "ak_test", absolutePath: file, toolSlug: "GMAIL_SEND_EMAIL", fetchImpl: async () => { @@ -121,7 +124,7 @@ test("a failed storage PUT is reported as its own step", async (t) => { const file = await scratchFile(t, "x.pdf", "x"); await assert.rejects( stageFileForComposio({ - apiKey: "ck", + apiKey: "ak_test", absolutePath: file, toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", fetchImpl: async (url) => (String(url).includes("/files/upload/request") @@ -135,7 +138,7 @@ test("a failed storage PUT is reported as its own step", async (t) => { test("a response with no storage key is an error, not a silent success", async (t) => { const file = await scratchFile(t, "x.pdf", "x"); await assert.rejects( - stageFileForComposio({ apiKey: "ck", absolutePath: file, toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", fetchImpl: async () => ok({ id: "req" }) }), + stageFileForComposio({ apiKey: "ak_test", absolutePath: file, toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", fetchImpl: async () => ok({ id: "req" }) }), /no storage key/, ); }); @@ -145,7 +148,7 @@ test("staging refuses an oversize file before any network call", async (t) => { let called = false; await assert.rejects( stageFileForComposio({ - apiKey: "ck", + apiKey: "ak_test", absolutePath: file, toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", maxBytes: 1024, @@ -185,23 +188,38 @@ test("unknown extensions fall back to octet-stream rather than guessing", () => // A fake Composio MCP endpoint: answers initialize, swallows the notification, and executes the // staging code's two shapes (append a base64 chunk / verify + mint a key) against an in-memory -// sandbox, the way the real workbench's stdout reports them. -function fakeWorkbench({ failWith = "", corrupt = false } = {}) { +// sandbox, the way the real workbench's stdout reports them. `sse` answers as server-sent events +// with a server-to-client ping (it has an id AND a method) ahead of the real answer. +function fakeWorkbench({ failWith = "", corrupt = false, sse = false, isError = false, rpcError = false, httpStatus = 0, echoKeyInError = false } = {}) { const calls = []; const files = new Map(); const fetchImpl = async (url, init) => { + if (init.method === "DELETE") { + calls.push({ url, method: "DELETE", headers: init.headers }); + return { ok: true, status: 200, headers: { get: () => null }, text: async () => "" }; + } const rpc = JSON.parse(init.body); - calls.push({ url, key: init.headers["x-consumer-api-key"], rpc }); - const headers = new Map([["mcp-session-id", "wb-session"]]); - const reply = (payload) => ({ ok: true, status: 200, headers: { get: (h) => headers.get(h) || null }, text: async () => (payload === null ? "" : JSON.stringify(payload)) }); - if (rpc.id === undefined) return reply(null); - if (rpc.method !== "tools/call") return reply({ jsonrpc: "2.0", id: rpc.id, result: { capabilities: {} } }); - assert.equal(rpc.params.name, "COMPOSIO_REMOTE_WORKBENCH"); + calls.push({ url, method: "POST", headers: init.headers, key: init.headers["x-consumer-api-key"], rpc }); + const respond = (payload) => { + const body = payload === null ? "" : sse + ? `event: message\ndata: ${JSON.stringify({ jsonrpc: "2.0", id: payload.id, method: "ping" })}\n\nevent: message\ndata: ${JSON.stringify(payload)}\n\n` + : JSON.stringify(payload); + return { ok: true, status: 200, headers: { get: (h) => (h === "mcp-session-id" ? "wb-session" : null) }, text: async () => body }; + }; + if (rpc.id === undefined) return respond(null); + if (rpc.method !== "tools/call") return respond({ jsonrpc: "2.0", id: rpc.id, result: { capabilities: {} } }); + if (httpStatus) return { ok: false, status: httpStatus, headers: { get: () => null }, text: async () => `upstream said no to ${echoKeyInError ? init.headers["x-consumer-api-key"] : "you"}` }; + if (rpcError) return respond({ jsonrpc: "2.0", id: rpc.id, error: { code: -32000, message: "tool unavailable" } }); + const answer = (text, extra = {}) => respond({ jsonrpc: "2.0", id: rpc.id, result: { content: [{ type: "text", text }], ...extra } }); + if (isError) return answer("upstream tool exploded", { isError: true }); + if (failWith !== "") { + const error = echoKeyInError ? `${failWith} (key ${init.headers["x-consumer-api-key"]})` : failWith; + return answer(JSON.stringify({ data: { stdout: "", error }, successful: false })); + } const code = rpc.params.arguments.code_to_execute; const target = code.match(/open\('([^']+)'/)[1]; let stdout = ""; const chunk = code.match(/b64decode\('([^']*)'\)/); - if (failWith) return reply({ jsonrpc: "2.0", id: rpc.id, result: { content: [{ type: "text", text: JSON.stringify({ data: { stdout: "", error: failWith }, successful: false }) }] } }); if (chunk) { const prior = code.includes("'wb'") ? Buffer.alloc(0) : files.get(target) || Buffer.alloc(0); files.set(target, Buffer.concat([prior, Buffer.from(chunk[1], "base64")])); @@ -211,84 +229,176 @@ function fakeWorkbench({ failWith = "", corrupt = false } = {}) { const md5 = createHash("md5").update(data).digest("hex"); stdout = `CGSTAGE${JSON.stringify({ md5, bytes: data.length, s3key: `wb/${path.basename(target)}` })}\n`; } - return reply({ jsonrpc: "2.0", id: rpc.id, result: { content: [{ type: "text", text: JSON.stringify({ data: { stdout, error: "" }, successful: true }) }] } }); + return answer(JSON.stringify({ data: { stdout, error: "" }, successful: true })); }; - return { calls, files, fetchImpl }; + const toolCalls = () => calls.filter((call) => call.rpc?.method === "tools/call"); + return { calls, files, fetchImpl, toolCalls }; } -test("a consumer key stages through the MCP workbench in chunks and never touches the REST API", async (t) => { +const MCP = "https://mcp.example.test/mcp"; +const stage = (overrides) => stageFileForComposio({ apiKey: "ck_consumer", toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", mcpUrl: MCP, ...overrides }); + +test("a consumer key stages through the MCP workbench in chunks, in ONE session, and never touches REST", async (t) => { const contents = Buffer.from("0123456789abcdefghij"); // 20 bytes → 3 chunks of 8 const file = await scratchFile(t, "report.pdf", contents); const wb = fakeWorkbench(); - const staged = await stageFileForComposio({ - apiKey: "ck_consumer", - absolutePath: file, - toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", - mcpUrl: "https://mcp.example.test/mcp", - workbenchChunkBytes: 8, - fetchImpl: wb.fetchImpl, - }); + const staged = await stage({ absolutePath: file, workbenchChunkBytes: 8, fetchImpl: wb.fetchImpl }); assert.equal(staged.route, "workbench"); assert.deepEqual(staged.file, { name: "report.pdf", mimetype: guessMimeType("report.pdf"), s3key: "wb/report.pdf" }); - assert.ok(wb.calls.every((call) => call.url === "https://mcp.example.test/mcp" && call.key === "ck_consumer")); - const codes = wb.calls.filter((call) => call.rpc.method === "tools/call").map((call) => call.rpc.params.arguments.code_to_execute); + assert.ok(wb.calls.every((call) => call.url === MCP && call.headers["x-consumer-api-key"] === "ck_consumer")); + // Every request after initialize carries the session the server minted: the sandbox only keeps + // the appended chunks for the life of that one session. + const afterInit = wb.calls.slice(1); + assert.ok(afterInit.length >= 5 && afterInit.every((call) => call.headers["mcp-session-id"] === "wb-session"), "the session id is sent back on every later request"); + const codes = wb.toolCalls().map((call) => call.rpc.params.arguments.code_to_execute); assert.equal(codes.length, 4, "three chunks plus the verification call"); assert.match(codes[0], /'wb'/); assert.ok(codes.slice(1, 3).every((code) => /'ab'/.test(code)), "later chunks append"); assert.deepEqual([...wb.files.values()][0], contents, "the sandbox holds exactly the file's bytes"); assert.match(codes[3], /get_mount_file_s3_key/); + // The session is ended; the sandbox copy is not deleted (it IS the s3key's storage). + assert.equal(wb.calls.at(-1).method, "DELETE"); + assert.ok(!codes.some((code) => /os\.remove|unlink|rmtree/.test(code))); +}); + +test("zero-length files and exact chunk multiples both reassemble exactly", async (t) => { + for (const [label, contents, chunks] of [["empty", Buffer.alloc(0), 1], ["exact", Buffer.from("abcdefgh12345678"), 2]]) { + const file = await scratchFile(t, `${label}.txt`, contents); + const wb = fakeWorkbench(); + const staged = await stage({ absolutePath: file, workbenchChunkBytes: 8, fetchImpl: wb.fetchImpl }); + assert.equal(staged.bytes, contents.length, label); + assert.equal(wb.toolCalls().length, chunks + 1, `${label}: ${chunks} write call(s) + verify`); + assert.deepEqual([...wb.files.values()][0], contents, label); + } +}); + +test("server-sent events are parsed, and a server ping is never taken for the answer", async (t) => { + const file = await scratchFile(t, "sse.txt", "hello"); + const wb = fakeWorkbench({ sse: true }); + const staged = await stage({ absolutePath: file, fetchImpl: wb.fetchImpl }); + assert.equal(staged.file.s3key, "wb/sse.txt"); +}); + +test("the bytes come from the proven descriptor, even if the path is swapped for a symlink", async (t) => { + const dir = await mkdtemp(path.join(os.tmpdir(), "gateway-composio-race-")); + t.after(() => rm(dir, { recursive: true, force: true })); + const inside = path.join(dir, "deliverable.txt"); + const outside = path.join(dir, "operator-secret.txt"); + await writeFile(inside, "the deliverable\n"); + await writeFile(outside, "OPERATOR SECRET\n"); + const handle = await open(inside, constants.O_RDONLY | constants.O_NOFOLLOW); + t.after(() => handle.close().catch(() => {})); + // The race the confinement proof is meant to win: after the proof, the container replaces the + // proven name with a symlink to a file outside the folder. + await unlink(inside); + await symlink(outside, inside); + const wb = fakeWorkbench(); + const staged = await stage({ handle, filename: "deliverable.txt", fetchImpl: wb.fetchImpl }); + assert.deepEqual([...wb.files.values()][0], Buffer.from("the deliverable\n"), "what was proven is what was sent"); + assert.equal(staged.bytes, 16); + // A direct caller handing a path gets O_NOFOLLOW: a symlink is refused, not followed. + await assert.rejects(stage({ absolutePath: inside, fetchImpl: wb.fetchImpl }), /ELOOP|symbolic link/i); +}); + +test("a file that grows past the cap after the size check is refused, not read whole", async () => { + let reads = 0; + const growing = { + stat: async () => ({ isFile: () => true, size: 4 }), + read: async (buffer) => { + reads += 1; + buffer.fill(0x61, 0, 16); + return { bytesRead: 16 }; + }, + }; + const wb = fakeWorkbench(); + await assert.rejects(stage({ handle: growing, filename: "grow.bin", workbenchMaxBytes: 40, fetchImpl: wb.fetchImpl }), /staging limit is 40/); + assert.ok(reads <= 3, "reading stops one chunk past the limit"); + assert.equal(wb.calls.length, 0, "nothing was sent"); }); test("the sandbox path cannot carry code: the file name is reduced to safe characters", async (t) => { const file = await scratchFile(t, "plain.txt", "x"); const wb = fakeWorkbench(); - const staged = await stageFileForComposio({ - apiKey: "ck_consumer", - absolutePath: file, - toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", - filename: "q'); import os; os.system('id') #.txt", - mcpUrl: "https://mcp.example.test/mcp", - fetchImpl: wb.fetchImpl, - }); - const code = wb.calls.find((call) => call.rpc.method === "tools/call").rpc.params.arguments.code_to_execute; + const staged = await stage({ absolutePath: file, filename: "q'); import os; os.system('id') #.txt", fetchImpl: wb.fetchImpl }); + const code = wb.toolCalls()[0].rpc.params.arguments.code_to_execute; assert.ok(!code.includes("import os; os.system"), "the caller's name never reaches the Python source"); assert.match(code, /channelgate-stage\/[0-9a-f]{16}\/q_import_os_os.system_id_.txt'/); - // The FileUploadable still carries the name the caller asked for. - assert.equal(staged.file.name, "q'); import os; os.system('id') #.txt"); + assert.equal(staged.file.name, "q'); import os; os.system('id') #.txt", "the FileUploadable keeps the real name"); }); -test("an incomplete transfer is refused rather than staged", async (t) => { +test("every workbench failure mode is an error, and none carries the key", async (t) => { const file = await scratchFile(t, "data.csv", "a,b\n1,2\n"); - const wb = fakeWorkbench({ corrupt: true }); - await assert.rejects( - stageFileForComposio({ apiKey: "ck_consumer", absolutePath: file, toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", mcpUrl: "https://mcp.example.test/mcp", fetchImpl: wb.fetchImpl }), - /incomplete; nothing was staged/, - ); + const cases = [ + [{ corrupt: true }, /incomplete; nothing was staged/], + [{ failWith: "PermissionError: /mnt/files is read-only", echoKeyInError: true }, /PermissionError/], + [{ failWith: "" }, null], + [{ isError: true }, /upstream tool exploded/], + [{ rpcError: true }, /tool unavailable/], + [{ httpStatus: 500, echoKeyInError: true }, /HTTP 500/], + ]; + for (const [options, pattern] of cases) { + if (options.failWith === "") continue; // an empty error with successful:true is a success; covered above + const wb = fakeWorkbench(options); + await assert.rejects(stage({ absolutePath: file, fetchImpl: wb.fetchImpl }), (error) => { + assert.match(error.message, pattern, JSON.stringify(options)); + assert.ok(!error.message.includes("ck_consumer"), `no key in: ${error.message}`); + return true; + }); + } + // successful:false with NO error text still fails. + const silent = fakeWorkbench(); + const failing = async (url, init) => { + const reply = await silent.fetchImpl(url, init); + const rpc = init.body ? JSON.parse(init.body) : {}; + if (rpc.method !== "tools/call") return reply; + return { ...reply, text: async () => JSON.stringify({ jsonrpc: "2.0", id: rpc.id, result: { content: [{ type: "text", text: JSON.stringify({ data: { stdout: "", error: "" }, successful: false }) }] } }) }; + }; + await assert.rejects(stage({ absolutePath: file, fetchImpl: failing }), /the sandbox reported a failure/); }); -test("a sandbox error surfaces as an error, without the key", async (t) => { - const file = await scratchFile(t, "data.csv", "a,b\n"); - const wb = fakeWorkbench({ failWith: "PermissionError: /mnt/files is read-only" }); +test("an upstream body that echoes the key is scrubbed on the REST route too", async (t) => { + const file = await scratchFile(t, "x.pdf", "%PDF"); await assert.rejects( - stageFileForComposio({ apiKey: "ck_consumer", absolutePath: file, toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", mcpUrl: "https://mcp.example.test/mcp", fetchImpl: wb.fetchImpl }), - (error) => /PermissionError/.test(error.message) && !error.message.includes("ck_consumer"), + stageFileForComposio({ + apiKey: "ak_project_key_value", + absolutePath: file, + toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", + apiBase: "https://backend.example.test", + fetchImpl: async () => ({ ok: false, status: 401, text: async () => `{"message":"Invalid API key: ak_project_key_value"}` }), + }), + (error) => /HTTP 401/.test(error.message) && !error.message.includes("ak_project_key_value") && /\[redacted\]/.test(error.message), ); }); +test("a hung workbench is bounded by the overall deadline", async (t) => { + const file = await scratchFile(t, "slow.txt", "slow"); + const hanging = (url, init) => new Promise((_, reject) => { + init.signal?.addEventListener("abort", () => reject(init.signal.reason), { once: true }); + }); + // AbortSignal.timeout's timer is unref'd: in the daemon the event loop is always alive, but a + // lone test process would exit before it fired. Hold the loop open for the test's duration. + const keepAlive = setInterval(() => {}, 1000); + try { + await assert.rejects(stage({ absolutePath: file, fetchImpl: hanging, workbenchDeadlineMs: 50 }), /timed out/); + } finally { + clearInterval(keepAlive); + } +}); + test("a consumer key has its own, lower size cap, checked before any call", async (t) => { const file = await scratchFile(t, "big.bin", Buffer.alloc(64)); const wb = fakeWorkbench(); - await assert.rejects( - stageFileForComposio({ apiKey: "ck_consumer", absolutePath: file, toolSlug: "GOOGLEDRIVE_UPLOAD_FILE", mcpUrl: "https://mcp.example.test/mcp", workbenchMaxBytes: 32, fetchImpl: wb.fetchImpl }), - /staging limit is 32 with a Composio consumer key/, - ); + await assert.rejects(stage({ absolutePath: file, workbenchMaxBytes: 32, fetchImpl: wb.fetchImpl }), /staging limit is 32 with a Composio consumer key/); assert.equal(wb.calls.length, 0); }); -test("consumer and project keys are told apart by prefix", () => { +test("only a known project key takes the REST route; everything else is a consumer key", () => { + assert.equal(isProjectApiKey("ak_abc"), true); assert.equal(isConsumerKey("ck_abc"), true); + assert.equal(isConsumerKey("some-unprefixed-token"), true, "an unknown shape goes where stored tokens work"); assert.equal(isConsumerKey("ak_abc"), false); assert.equal(isConsumerKey(""), false); assert.equal(sandboxFileName("../../etc/passwd"), "etc_passwd"); assert.equal(sandboxFileName(""), "file"); + assert.equal(scrubStagingText("key ck_secret1 and " + "A".repeat(60), "ck_secret1"), "key [redacted] and [data]"); }); From db81c7b354067e3e8e36b130e6409ae681bf3cbf Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 00:41:28 +0300 Subject: [PATCH 04/14] test: prove the staging tool hands over the proven descriptor, not a path Review round 2 (approve with nits) found that reverting the TOOL to close the proven handle and pass a path still left every test green: O_NOFOLLOW catches a swapped final component, but not a swapped parent directory. The tool now takes an injectable stageFile (production uses the real one) and a test asserts staging receives a live descriptor of the proven file and no path; that revert now fails it. Also: the comment about sandbox lifetime no longer contradicts the one about keeping the staged copy; the dead failure-table row is gone; and FEATURES records that on composio-agent backed by the org key the kept copy is listable from any conversation sharing that key. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- FEATURES.md | 5 ++++- src/gateway/composio-files.js | 14 ++++++++------ src/mcp/tools/file-sharing.js | 5 ++++- test/composio-files.test.js | 2 -- test/file-sharing-tools.test.js | 25 +++++++++++++++++++++++-- 5 files changed, 39 insertions(+), 12 deletions(-) diff --git a/FEATURES.md b/FEATURES.md index 1dd8351..dca7083 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -1819,7 +1819,10 @@ A categorized catalog of what's shipped. Cross-linked to `TEST-PLAN.md` checks. characters, so a file name can never become Python source. Consumer-key staging is capped at 25 MB (project keys keep 100 MB), within an overall 10-minute deadline; the MCP session is ended afterwards, but the sandbox copy is kept because the s3key IS that file's storage (deleting it - breaks the upload — verified live). The bytes are read from the SAME descriptor the confinement + breaks the upload — verified live). A known property of this route: the copy stays in that + identity's Composio file storage, so on `composio-agent` backed by the organization key it is + listable from any conversation sharing that key (its random directory name is not a secret) — + stage on `composio-user`, or give the channel its own Composio key, when that matters. The bytes are read from the SAME descriptor the confinement check proved (O_NOFOLLOW + /proc/self/fd), bounded to the cap: the tool used to close that descriptor and reopen the file by path, so a symlink swapped in by the container between proof and read could have sent any file the daemon can read. Every error text is scrubbed of the key diff --git a/src/gateway/composio-files.js b/src/gateway/composio-files.js index a0bf419..bbd393c 100644 --- a/src/gateway/composio-files.js +++ b/src/gateway/composio-files.js @@ -80,15 +80,17 @@ function failure(step, status, body) { // // It still runs in the daemon, for the same reason as the REST route: the key never reaches the // model, and neither do the file's bytes. They cross as base64 inside the code the daemon sends, -// in chunks, because one request above ~5 MB of base64 is rejected (413); the sandbox keeps its -// files for the life of one MCP session, so the chunks append to one file and a final call checks -// the md5 before minting the key. That makes this route slower than the REST one, hence its own, -// lower size cap and an overall deadline. +// in chunks, because one request above ~5 MB of base64 is rejected (413). All chunk calls go +// through ONE MCP session, so they land in the same sandbox and append to one file; a final call +// checks the md5 before minting the key. That makes this route slower than the REST one, hence its +// own, lower size cap and an overall deadline. // // The sandbox copy is deliberately NOT deleted afterwards: the s3key IS that mounted file's // storage, and removing the file makes the upload fail with "the file does not exist in storage" -// (verified live). It lives in the same identity's own Composio sandbox as any workbench file. The -// MCP session itself is ended (HTTP DELETE) — that does not affect the key (also verified). +// (verified live). Ending the MCP session (HTTP DELETE) does not affect the file or the key (also +// verified). So the copy stays in that identity's Composio file storage — and on `composio-agent` +// backed by the organization key, that storage is shared by every conversation using the same +// key, whose own workbench can list it. The random directory name is not a secret. export const COMPOSIO_WORKBENCH_STAGE_MAX_BYTES = 25 * 1024 * 1024; export const COMPOSIO_WORKBENCH_CHUNK_BYTES = 768 * 1024; export const COMPOSIO_WORKBENCH_DEADLINE_MS = 10 * 60_000; diff --git a/src/mcp/tools/file-sharing.js b/src/mcp/tools/file-sharing.js index d94699f..d8d9270 100644 --- a/src/mcp/tools/file-sharing.js +++ b/src/mcp/tools/file-sharing.js @@ -76,6 +76,9 @@ export async function resolveComposioKey({ identity, authorId, meta }) { export function register(server, ctx) { const { slug, channelId, createdBy, text, loadMeta } = ctx; + // Injectable only so the tests can prove what the tool hands to staging (a proven descriptor, + // never a path to reopen); production always uses the real function. + const stageFile = ctx.stageFile || stageFileForComposio; server.registerTool( "stage_file_for_composio", @@ -131,7 +134,7 @@ export function register(server, ctx) { // file by path. The container can write this folder, so a path reopened after the proof // could by then be a symlink into the operator's home, read by the unsandboxed daemon. handle = opened.handle; - const staged = await stageFileForComposio({ + const staged = await stageFile({ apiKey: key, handle, toolSlug: tool, diff --git a/test/composio-files.test.js b/test/composio-files.test.js index c436c98..069ba1a 100644 --- a/test/composio-files.test.js +++ b/test/composio-files.test.js @@ -331,13 +331,11 @@ test("every workbench failure mode is an error, and none carries the key", async const cases = [ [{ corrupt: true }, /incomplete; nothing was staged/], [{ failWith: "PermissionError: /mnt/files is read-only", echoKeyInError: true }, /PermissionError/], - [{ failWith: "" }, null], [{ isError: true }, /upstream tool exploded/], [{ rpcError: true }, /tool unavailable/], [{ httpStatus: 500, echoKeyInError: true }, /HTTP 500/], ]; for (const [options, pattern] of cases) { - if (options.failWith === "") continue; // an empty error with successful:true is a success; covered above const wb = fakeWorkbench(options); await assert.rejects(stage({ absolutePath: file, fetchImpl: wb.fetchImpl }), (error) => { assert.match(error.message, pattern, JSON.stringify(options)); diff --git a/test/file-sharing-tools.test.js b/test/file-sharing-tools.test.js index 80de380..bd8cb6a 100644 --- a/test/file-sharing-tools.test.js +++ b/test/file-sharing-tools.test.js @@ -17,7 +17,7 @@ import assert from "node:assert/strict"; import http from "node:http"; import { createHash } from "node:crypto"; import path from "node:path"; -import { mkdirSync, writeFileSync } from "node:fs"; +import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; import { ensureTestEnv } from "./helpers.js"; @@ -82,7 +82,7 @@ process.env.COMPOSIO_API_BASE = `http://127.0.0.1:${composio.address().port}`; process.env.COMPOSIO_MCP_URL = `http://127.0.0.1:${composio.address().port}/mcp`; test.after(() => composio.close()); -function tools({ author = "U_AUTHOR", meta = {} } = {}) { +function tools({ author = "U_AUTHOR", meta = {}, stageFile } = {}) { const map = new Map(); register({ registerTool: (name, _schema, handler) => map.set(name, handler) }, { channelId: CHANNEL, @@ -90,6 +90,7 @@ function tools({ author = "U_AUTHOR", meta = {} } = {}) { createdBy: author, text: (t) => t, loadMeta: async () => ({ platform: "slack", isDM: false, ...meta }), + ...(stageFile ? { stageFile } : {}), }); return map; } @@ -123,6 +124,26 @@ test("a consumer key stages through the Composio workbench on the named identity assert.ok(staged.every((call) => call.key === "ck_channel")); }); +test("the tool hands staging the proven descriptor, never a path to reopen", async () => { + // The confinement proof is only worth something if the bytes come from the descriptor it proved: + // the container can write this folder, so a path reopened after the proof could by then be a + // symlink (or sit under a swapped parent directory) pointing anywhere the daemon can read. + await setUser("U_AUTHOR", { name: "Author", approved: true, composioToken: "ck_personal" }); + let seen = null; + const reply = await tools({ + stageFile: async (options) => { + const bytes = Buffer.alloc(64); + const { bytesRead } = await options.handle.read(bytes, 0, 64, 0); + seen = { ...options, content: bytes.subarray(0, bytesRead).toString(), fd: typeof options.handle?.fd }; + return { file: { name: "PROPOSAL.pdf", mimetype: "application/pdf", s3key: "k" }, bytes: bytesRead, deduplicated: false, tool: "GOOGLEDRIVE_UPLOAD_FILE", route: "workbench" }; + }, + }).get("stage_file_for_composio")({ path: "work/PROPOSAL.pdf", tool: "GOOGLEDRIVE_UPLOAD_FILE", identity: "user" }); + assert.match(reply, /Staged `work\/PROPOSAL\.pdf`/); + assert.equal(seen.fd, "number", "a live descriptor reaches staging"); + assert.equal(seen.absolutePath, undefined, "no path is handed over to be reopened"); + assert.equal(seen.content, readFileSync(path.join(WORKDIR, "work", "PROPOSAL.pdf"), "utf8"), "and it is the proven file"); +}); + test("a project API key keeps the REST upload route", async () => { staged.length = 0; await setUser("U_PROJECT", { name: "Project", approved: true, composioToken: "ak_project" }); From e55a3b082573f8cc9456eaff702277a94f81600e Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 00:43:56 +0300 Subject: [PATCH 05/14] fix: state the prune rule's consequence truthfully, keep the admin path Review round 2 (changes required): 'delete every channel's home' was false for three of the four commands - podman image prune -a never touches volumes, and system prune spares named volumes without --volumes. A model that knows podman could argue a false rule away. Each command CAN remove channel homes or the runtime image every channel runs on, so the rule now says that; it also names podman system reset, and keeps the documented removal path (runtime:storage -- --apply only when an admin asks) instead of contradicting it with 'report only'. The block fits its budget in the measured configuration (4,085 bytes); the intro, header note and network line were tightened without changing what they say. The worst switch combination (Admin + Auto, Lean off) was already 4,126 bytes before this rule and is now 4,120; a test stops it growing. The header note's three facts are now asserted. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- CHANGELOG.md | 8 ++++---- FEATURES.md | 10 ++++++---- TEST-PLAN.md | 8 +++++--- src/gateway/folders.js | 14 +++++++------- test/folders-generator-paths.test.js | 28 ++++++++++++++++++++++++---- 5 files changed, 46 insertions(+), 22 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5f32205..e05258b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,11 +19,11 @@ product overview. ## Unreleased - The assistant no longer suggests a blanket container prune when asked about disk space. The - rule against `podman system prune`, `podman image prune -a`, `podman volume prune` and + rule against `podman system prune`/`reset`, `podman image prune -a`, `podman volume prune` and `docker system prune` lived only in the operating guide, and a run that answered without opening - the guide recommended two of them — on this host they delete every channel's home. The rule now - sits among the few hard rules every run of every engine reads, pointing at the report-only - `npm run runtime:storage` instead. + the guide recommended two of them — on this host they can delete channel homes or the runtime + image. The rule now sits among the few hard rules every run of every engine reads, pointing at + `npm run runtime:storage` (removal only with `-- --apply` when an admin asks). ## 0.5.5 — 2026-09-24 diff --git a/FEATURES.md b/FEATURES.md index 8962bea..db953a5 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -2860,14 +2860,16 @@ are retired, bullet by bullet; everything else stands. "your X" never touches `composio-user`); that `COMPOSIO_MANAGE_CONNECTIONS` initiates connections rather than listing them; and that only the gateway's `run_in_background` / `run_agent_in_background` / `create_schedule` can report back after a turn ends; and that no run - runs or recommends a blanket container prune (`podman system prune`, `podman image prune -a`, - `podman volume prune`, `docker system prune`), which deletes every channel's home — disk cleanup - goes through the report-only `npm run runtime:storage` — stated here + runs or recommends `podman system prune`/`reset`, `podman image prune -a`, `podman volume prune` + or `docker system prune`, each of which can delete channel homes or the runtime image — disk + cleanup goes through `npm run runtime:storage`, with `-- --apply` only when an admin asks — stated here because a skill body is read only when the model opens it, and one engine reliably did not (retest, 2026-09-06; the prune rule after a 2026-09-25 run that skipped the skill and recommended both prunes). They ride clean mode too, are engine-neutral, and stay under 4 KB with the switches so the always-on prompt weight is read rather than skimmed - (4,091 bytes after the prune rule: the header note was tightened to make room, and the next + (4,085 bytes in the measured configuration after the prune rule — the header note, the rules' + intro and the network line were tightened to make room; the longest switch combination, Admin + + Auto with Lean off, is 4,120 bytes, down from 4,126 before, and a test stops it growing. The next always-on rule has to trade space for it). Editable three ways: the admin UI Instructions tab (edits the real file; managed block shown read-only with a Settings link; hash-guarded against concurrent writes), by hand, or by asking the agent — the `update_channel_instructions` gateway MCP tool diff --git a/TEST-PLAN.md b/TEST-PLAN.md index d65ac7a..2f21972 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -192,10 +192,12 @@ ## Host container-storage housekeeping guidance - [x] `test/folders-generator-paths.test.js`: the managed CLAUDE.md block (read by every run of both - engines, skill opened or not) forbids running or recommending `podman system prune`, + engines, skill opened or not) forbids running or recommending `podman system prune`/`reset`, `podman image prune -a`, `podman volume prune` and `docker system prune`, each named whole on - one line, and names `npm run runtime:storage` (report only); the gateway-owned block stays - under 4 KB. Live: OPS-DISK-01 (Airtable) on both engines — a 2026-09-25 Claude run never + one line, with a consequence true for all of them ("can delete channel homes or the runtime + image") and the admin path `npm run runtime:storage` (`-- --apply` only if an admin asks); the + header note keeps its three facts; the block stays under 4 KB in the measured configuration + and the worst switch combination may not exceed its current 4,120 bytes. Live: OPS-DISK-01 (Airtable) on both engines — a 2026-09-25 Claude run never opened the skill and recommended both prunes; the Codex run read administration.md and did not. - [x] `test/host-housekeeping-guide.test.js`: the materialized guide for every platform routes disk/stale-container/old-image questions to `references/administration.md` and carries the diff --git a/src/gateway/folders.js b/src/gateway/folders.js index e7dd2b4..8b02222 100644 --- a/src/gateway/folders.js +++ b/src/gateway/folders.js @@ -105,7 +105,7 @@ function stripBlock(content, start, end) { const GW_START = ""; const GW_END = ""; const GW_NOTE = `> ⚙️ Gateway-managed block: do NOT edit between these markers; the gateway rewrites it. Below the -> end marker are this channel's own standing instructions, never overwritten. To add one when +> end marker: this channel's own standing instructions, never overwritten. To add one when > asked, use \`update_channel_instructions\` (or edit this file if writable).`; // What each mode actually grants, in the agent's own terms — the label alone ("Bash") does not @@ -134,7 +134,7 @@ export function channelSwitchesNote(meta = {}) { ? "**on** — this conversation is meant to use the internet. There is no per-domain allow-list." : network === "unsupported" ? "requested **on**, but this conversation's engine cannot run with the network on — treat it as off." - : `**off** — this conversation is NOT meant to use the internet: don't fetch, install, push or call out, and say the switch is off instead of trying. ${NETWORK_POLICY_ENFORCED ? "" : `The switch is ${NETWORK_ADVISORY_NOTE}, so a request may still succeed — that is not permission.`}`.trim(); + : `**off** — this conversation is NOT meant to use the internet: don't fetch, install, push or call out; say the switch is off instead. ${NETWORK_POLICY_ENFORCED ? "" : `The switch is ${NETWORK_ADVISORY_NOTE}, so a request may still succeed — that is not permission.`}`.trim(); return [ "**This conversation's switches** (an admin sets them; they apply from the next message):", `- Mode: **${mode}** — ${MODE_NOTE[mode]}.`, @@ -187,8 +187,8 @@ export function channelSwitchesNote(meta = {}) { // while the Codex run, which read administration.md, warned against exactly those. On this host a // blanket prune deletes every stopped channel's HOME volume (sessions, CLI logins, memory), so the // one-line core belongs here. -const HARD_RULES = `**Hard rules (not optional)** — they apply wherever the named tools exist; the reasoning and the -tool shapes are in the \`gateway-usage\` skill: +const HARD_RULES = `**Hard rules (not optional)** — they apply wherever the named tools exist; details are in the +\`gateway-usage\` skill: - Use \`ask_questions\` for clarification. - **Two Composio identities.** \`composio-user\` = the REQUESTER's own accounts; \`composio-agent\` = the shared agent's own (either may appear with \`_\` for \`-\`). Reads and searches may use either or @@ -220,9 +220,9 @@ tool shapes are in the \`gateway-usage\` skill: one of these too: \`create_schedule\` (or \`run_agent_in_background\` for a self-contained watcher), never an in-turn sleep/poll loop, a \`Monitor\`-style wait, or a harness background task — even when the loop would finish inside this turn. -- **Never run or recommend a blanket prune** — \`podman system prune\`, \`podman image prune -a\`, - \`podman volume prune\`, \`docker system prune\` delete every channel's home. Cleanup: - \`npm run runtime:storage\` (report only).`; +- **Never run or recommend \`podman system prune\`/\`reset\`, \`podman image prune -a\`, \`podman volume prune\` + or \`docker system prune\`**: they can delete channel homes or the runtime image. Cleanup: + \`npm run runtime:storage\` (\`-- --apply\` only if an admin asks).`; // Compose the managed block for a channel: the do-not-edit note, this conversation's switches, the // hard rules, and (outside clean mode) the admin's global instructions. Deliberately nothing about diff --git a/test/folders-generator-paths.test.js b/test/folders-generator-paths.test.js index 8f7deeb..c633151 100644 --- a/test/folders-generator-paths.test.js +++ b/test/folders-generator-paths.test.js @@ -241,13 +241,24 @@ test("the managed block carries the hard rules a run must never get wrong", () = assert.match(block, /A bounded "check every N minutes, K times"/); assert.match(block, /never an in-turn sleep\/poll loop/); - // 5. No blanket container prune, ever — in the block every run loads, not only in the skill - // (OPS-DISK-01: a Claude run never opened the skill and recommended `podman system prune`). - assert.match(block, /Never run or recommend a blanket prune/); + // 5. No blanket container prune or reset, ever — in the block every run loads, not only in the + // skill (OPS-DISK-01: a Claude run never opened the skill and recommended `podman system prune`). + // The stated consequence must be TRUE for every command named (a model that knows podman can + // argue away a false one): not all of them touch volumes, but each can remove channel homes + // or the runtime image. + assert.match(block, /Never run or recommend `podman system prune`\/`reset`/); for (const command of ["podman system prune", "podman image prune -a", "podman volume prune", "docker system prune"]) { assert.ok(block.includes(`\`${command}\``), `${command} is named whole, on one line`); } - assert.ok(block.includes("`npm run runtime:storage` (report only)")); + assert.match(block, /they can delete channel homes or the runtime image/); + assert.doesNotMatch(block, /delete every channel's home/); + // The admin-approved removal path stays open. + assert.ok(block.includes("`npm run runtime:storage` (`-- --apply` only if an admin asks)")); + + // The header note keeps its three facts through any future squeeze for budget. + assert.match(block, /do NOT edit between these markers/); + assert.match(block, /never overwritten/); + assert.match(block, /`update_channel_instructions`/); // Still the whole block, not a replacement for it: tonight's switches section survives. assert.match(block, /This conversation's switches/); @@ -262,4 +273,13 @@ test("the managed block carries the hard rules a run must never get wrong", () = // future rule needs more than this, it belongs in the skill, not here. const owned = gatewayInstructionsBlock({ allowBash: true, cleanMode: true }); assert.ok(Buffer.byteLength(owned, "utf8") < 4096, `managed block is ${Buffer.byteLength(owned, "utf8")} bytes — keep it under 4 KB`); + // The budget above is measured in one representative configuration. The longest switch lines + // (Admin + Auto, Lean off) already push the worst case slightly past 4 KB; it must not grow. + const keys = ["adminMode", "allowBash", "autoMode", "cleanMode", "allowNetwork", "isDM"]; + let worst = 0; + for (let mask = 0; mask < 1 << keys.length; mask++) { + const meta = Object.fromEntries(keys.map((key, i) => [key, Boolean(mask & (1 << i))])); + worst = Math.max(worst, Buffer.byteLength(gatewayInstructionsBlock(meta), "utf8")); + } + assert.ok(worst <= 4120, `worst-case managed block is ${worst} bytes — it may shrink, never grow`); }); From dec0e1cf44ee890ff5023241761b8ab2fb90d4e9 Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 00:49:49 +0300 Subject: [PATCH 06/14] docs: give the guide the same true prune consequence as the hard rule Review round 3 (approve with nits): the skill's routing row and administration.md still said the prunes 'delete every channel HOME volume', the claim the hard rule just dropped for being false for some commands - so a run that opened the skill read text contradicting the rule. Both now name the same five commands and the same consequence: they can delete channel HOME volumes or the runtime image. The code comment above the hard rules says the same. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- src/gateway/folders.js | 6 +++--- src/gateway/gateway-usage/SKILL.md | 2 +- src/gateway/gateway-usage/references/administration.md | 9 +++++---- test/host-housekeeping-guide.test.js | 8 ++++++-- 4 files changed, 15 insertions(+), 10 deletions(-) diff --git a/src/gateway/folders.js b/src/gateway/folders.js index 8b02222..acda131 100644 --- a/src/gateway/folders.js +++ b/src/gateway/folders.js @@ -184,9 +184,9 @@ export function channelSwitchesNote(meta = {}) { // administration.md already forbade suggesting a blanket prune after 0.5.3's L3 failure — but that // text lives in the skill, and a Claude run asked "what can we clean up?" never opened it, measured // the disk with its own tools and recommended `podman image prune -a` and `podman system prune`, -// while the Codex run, which read administration.md, warned against exactly those. On this host a -// blanket prune deletes every stopped channel's HOME volume (sessions, CLI logins, memory), so the -// one-line core belongs here. +// while the Codex run, which read administration.md, warned against exactly those. On this host +// those commands can delete channel HOME volumes (sessions, CLI logins, memory) or the runtime image +// every channel needs, so the one-line core belongs here. const HARD_RULES = `**Hard rules (not optional)** — they apply wherever the named tools exist; details are in the \`gateway-usage\` skill: - Use \`ask_questions\` for clarification. diff --git a/src/gateway/gateway-usage/SKILL.md b/src/gateway/gateway-usage/SKILL.md index 66dce07..2508896 100644 --- a/src/gateway/gateway-usage/SKILL.md +++ b/src/gateway/gateway-usage/SKILL.md @@ -176,7 +176,7 @@ credential or connection is needed, without exposing its value. | See, grant or remove skills here, apply a skills template, create/update/propose a skill, see skill usage | `references/skills.md` | `gateway` → `show_channel_skills`, `add_channel_skills`, `apply_skill_template`, `create_skill`, `propose_skill_change`, `skill_usage_report` | | Change a channel/gateway setting, tokens, update/restart, or this guide | `references/administration.md` | `gateway` → `set_channel_*`, `set_my_*_token`, `update_gateway`, `restart_gateway`, `update_gateway_guide` | | Register an SSH key, grant/revoke SSH into this channel's container, get the connection block | `references/administration.md` | `gateway` → `add_my_ssh_key`, `grant_channel_ssh`, `revoke_channel_ssh`, `show_channel_ssh` | -| Check disk space, stale containers or old runtime images | `references/administration.md` | Host `/sudo` thread: `npm run runtime:storage` (reports, changes nothing). **Never run or suggest `podman system prune`, `podman volume prune` or `podman image prune -a`** — they delete channel HOME volumes (engine sessions, CLI logins). Report and ask; remove only via `-- --apply` when an admin says so | +| Check disk space, stale containers or old runtime images | `references/administration.md` | Host `/sudo` thread: `npm run runtime:storage` (reports, changes nothing). **Never run or suggest `podman system prune`, `podman system reset`, `podman volume prune`, `podman image prune -a` or `docker system prune`** — they can delete channel HOME volumes (engine sessions, CLI logins) or the runtime image. Report and ask; remove only via `-- --apply` when an admin says so | ## Tool identities: the bot, YOUR account, and the requester's account diff --git a/src/gateway/gateway-usage/references/administration.md b/src/gateway/gateway-usage/references/administration.md index 6f6251e..387422d 100644 --- a/src/gateway/gateway-usage/references/administration.md +++ b/src/gateway/gateway-usage/references/administration.md @@ -354,10 +354,11 @@ was created from alive. A full disk takes the daemon, its database and every cha so check before it gets there. **Never delete anything automatically.** Report what you found and what it would free, then let an -admin decide. And never *recommend* a blanket prune either: `podman system prune` -(with or without `--volumes`), `podman volume prune` and `podman image prune -a` look like the -routine fix, but they delete every channel HOME volume whose container happens to be stopped or -gone — its engine sessions, CLI logins and installed tools, unrecoverably. The safe path is always +admin decide. And never *recommend* a blanket prune either: `podman system prune` (`-a`, +`--volumes`), `podman system reset`, `podman volume prune`, `podman image prune -a` and +`docker system prune` look like the routine fix, but between them they can delete the HOME volume +of any channel whose container is stopped or gone — its engine sessions, CLI logins and installed +tools, unrecoverably — and the runtime image every channel needs. The safe path is always `npm run runtime:storage`, which knows which volumes are channel homes. Removal happens only on an explicit request or from a schedule an admin set up — not as a tidy-up you decided was helpful. diff --git a/test/host-housekeeping-guide.test.js b/test/host-housekeeping-guide.test.js index 2680520..17c854b 100644 --- a/test/host-housekeeping-guide.test.js +++ b/test/host-housekeeping-guide.test.js @@ -84,8 +84,12 @@ test("the skill's routing row forbids suggesting a blanket prune and names the s const skill = await readFile(guidePath(cwd, "SKILL.md"), "utf8"); const row = skill.split("\n").find((line) => line.startsWith("| Check disk space")); assert.ok(row, `${platform}: the routing row exists`); - assert.match(row, /Never run or suggest `podman system prune`, `podman volume prune` or `podman image prune -a`/, platform); - assert.match(row, /delete channel HOME volumes/, platform); + for (const command of ["podman system prune", "podman system reset", "podman volume prune", "podman image prune -a", "docker system prune"]) { + assert.ok(row.includes(`\`${command}\``), `${platform}: ${command}`); + } + assert.match(row, /Never run or suggest/, platform); + // The same TRUE consequence as the hard rule (not every command touches volumes). + assert.match(row, /can delete channel HOME volumes .* or the runtime image/, platform); assert.match(row, /npm run runtime:storage/, platform); } } finally { await rm(cwd, { recursive: true, force: true }); } From 4865abdc836f2d8b21bc4702b24c674707800d49 Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 01:18:32 +0300 Subject: [PATCH 07/14] fix: name the channel's own harness default as its inherited model Slack Settings resolved an unset channel model through the gateway's default engine, so a Claude-pinned channel on a Codex-default gateway read "Inherited default (gpt-6-sol)" although runs used the Claude default. The label now follows run.js: the template model when it belongs to the channel's harness, else that harness's default. The web admin's unconfigured VPN row also stops appending the server's "VPN is not configured" next to the identical setup hint. Found in QA-0925 (SLK-SETTINGS-TPL-01, SLK-SETTINGS-VPN-01). Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- CHANGELOG.md | 4 ++++ FEATURES.md | 4 +++- TEST-PLAN.md | 7 +++++++ public/app.js | 6 +++++- src/slack/app.js | 11 +++++++++-- test/channel-settings-modal.test.js | 18 ++++++++++++++++++ test/channel-vpn-web.test.js | 8 ++++++-- 7 files changed, 52 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6ebb2db..165452a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,10 @@ product overview. ## Unreleased +- Slack **⚙️ Settings** no longer claims a channel pinned to Claude inherits the Codex default + model (or the reverse): an unset model or effort now names what the channel's own harness would + actually use. The web admin's VPN status also stops saying "not configured" twice. + - Sending a generated file to Drive, Gmail and other Composio tools works again — or rather, works for the first time on a gateway using personal Composio tokens. `stage_file_for_composio` sent the stored token to Composio's REST upload, which only accepts project API keys; the tokens stored diff --git a/FEATURES.md b/FEATURES.md index 6a285d2..34f8cb9 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -375,7 +375,9 @@ A categorized catalog of what's shipped. Cross-linked to `TEST-PLAN.md` checks. inside a thread, **that thread's own pins** (the per-thread overrides the `/model` wizard's "just this thread" scope and the `claude`/`codex` directive write, which beat the channel at run time). An unset field preselects the label of what it inherits — the gateway default, an org DM - template's value, or "Follow channel (…)" for a thread — so nothing reads as a blank; each + template's value, or "Follow channel (…)" for a thread — so nothing reads as a blank (an unset + model or effort names what the channel's OWN harness would use, so a Claude-pinned channel on a + Codex-default gateway reads the Claude default, never the Codex one); each scope's model and effort lists follow the harness THAT scope resolves to, so a Codex-pinned thread inside a Claude channel offers Codex models. Changing one field drops only the dependents it invalidates (a harness change always clears the model, and the effort when the new harness diff --git a/TEST-PLAN.md b/TEST-PLAN.md index 1bd274d..2592a53 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -1972,6 +1972,13 @@ Automated: `test/channel-memory.test.js`, `test/memory-search.test.js`, General Settings with its notice; and a channel with a provisioned VPN shows *Checking status…* replaced by the real state, while a channel without one shows *Not configured* immediately with no flicker. +- [x] Automated inherited-model label (engine-independent): with the gateway default harness + Codex, a channel pinned to Claude with no model shows *Inherited default ()*, + not the Codex default, and offers only Claude models + (`test/channel-settings-modal.test.js`). The web admin's unconfigured VPN row explains the + missing setup once, without the server's own "VPN is not configured. …" repeated + (`test/channel-vpn-web.test.js`, browser case; run it from a path without a dot-directory — + the static server 404s any path under `.worktrees/`). - [ ] Live resume round trip (engine-independent Slack UI case, run once per harness where the session is minted by that harness): in a disposable channel, send a message, then open **⚙️ Settings → Resume Session** from a reply inside that thread. Pass when the tab shows the diff --git a/public/app.js b/public/app.js index c28ad07..39e6c7b 100644 --- a/public/app.js +++ b/public/app.js @@ -1771,7 +1771,11 @@ function mountChannelVpnControls(card, channelId) { refresh.disabled = pending; if (snapshot) { const labels = { unconfigured: "Not configured", unavailable: "Unavailable", off: "Off", starting: "Starting", on: "Connected", stopping: "Stopping", failed: "Failed" }; - const parts = [labels[snapshot.state] || "Unknown", snapshot.message]; + // The state word, plus the server's message only where the word cannot say why on its own (a + // failure, an unavailable service) — for the other states it restates the label, and for an + // unconfigured VPN it repeated the hint below almost word for word. + const explained = ["failed", "unavailable"].includes(snapshot.state) ? snapshot.message : ""; + const parts = [labels[snapshot.state] || "Unknown", explained]; if (!snapshot.configured) parts.push("An administrator must import the VPN profile and prepare the channel’s VPN service first."); if (snapshot.missingSecrets?.length) parts.push(`Add in Environment: ${snapshot.missingSecrets.join(", ")}.`); if (snapshot.configured && !snapshot.allowNetwork) parts.push("Enable Network and save the channel before starting VPN."); diff --git a/src/slack/app.js b/src/slack/app.js index b7ba39a..26b19fd 100644 --- a/src/slack/app.js +++ b/src/slack/app.js @@ -684,12 +684,19 @@ export async function runtimeScopes(slug, meta, snapshot, threadKey) { const channelEngine = runtime.effectiveEngineId; const channelModel = runtime.configuredModel || getDefaultModel(channelEngine) || ""; const parent = inheritedChannelRuntime(meta); + // An unset MODEL falls back to what the harness that actually runs this channel would use: its + // template's model when that belongs to it, else THAT harness's gateway default. `parent.model` + // falls back to the GATEWAY engine's default, so under a channel pinned to the non-default + // harness it named the other harness's default ("Inherited default (gpt-6-sol)" on a Claude + // channel) — misleading, though the run itself used the right one. Effort follows the same rule. + const parentModel = [parent.model, getDefaultModel(channelEngine)].find((m) => m && modelBelongsToEngine(m, channelEngine)) || ""; + const parentEffort = parent.effort && effortBelongsToModel(parent.effort, channelEngine, runtime.configuredModel || parentModel) ? parent.effort : ""; const channel = { values: { engine: runtime.configuredEngineId, model: runtime.configuredModel, effort: runtime.configuredEffort }, inherited: { engine: `Inherited default (${engineLabel(parent.engine)})`, - model: parent.model ? `Inherited default (${parent.model})` : "Engine default", - effort: parent.effort ? `Inherited default (${parent.effort})` : "Engine default", + model: parentModel ? `Inherited default (${parentModel})` : "Engine default", + effort: parentEffort ? `Inherited default (${parentEffort})` : "Engine default", }, options: { engines, models: modelsForEngine(channelEngine), efforts: effortChoices(channelEngine, channelModel) }, }; diff --git a/test/channel-settings-modal.test.js b/test/channel-settings-modal.test.js index 6211512..4c3fbc8 100644 --- a/test/channel-settings-modal.test.js +++ b/test/channel-settings-modal.test.js @@ -680,6 +680,24 @@ test("a DM following an org template inherits that template's runtime, not the g } }); +test("a channel pinned to the non-default harness names THAT harness's default model as inherited", async () => { + const { saveSettings, getSettings } = await import("../src/config/settings.js"); + const before = getSettings(); + try { + // Gateway default engine is Codex; this channel is pinned to Claude with no model of its own. + await saveSettings({ ...before, engine: "codex", defaultClaudeModel: "opus[1m]", defaultCodexModel: "gpt-6-sol" }); + const meta = { engine: "claude", model: "", effort: "" }; + const scopes = await runtimeScopes("pinned-claude-channel", meta, { runtime: { ...snapshot.runtime, configuredEngineId: "claude", configuredModel: "", configuredEffort: "", effectiveEngineId: "claude" } }, ""); + // What a run here actually uses is Claude's default; the Codex default must not be shown as + // the inherited value of a Claude channel (QA-0925: "Inherited default (gpt-6-sol)"). + assert.equal(scopes.channel.inherited.model, "Inherited default (opus[1m])"); + assert.equal(scopes.channel.inherited.engine, "Inherited default (Codex)", "the ENGINE row still names what an unset engine inherits"); + assert.ok(scopes.channel.options.models.every((option) => !/^gpt-/.test(option.value)), "and the catalog is Claude's"); + } finally { + await saveSettings(before); + } +}); + test("a thread pick is validated against the harness the dropdown offered, not the channel's", async () => { const { saveSession } = await import("../src/gateway/sessions.js"); const { getThreadEngine, getThreadModel, getThreadEffort } = await import("../src/gateway/thread-engine.js"); diff --git a/test/channel-vpn-web.test.js b/test/channel-vpn-web.test.js index 88af240..7169816 100644 --- a/test/channel-vpn-web.test.js +++ b/test/channel-vpn-web.test.js @@ -108,7 +108,7 @@ test("browser VPN switch applies immediately, polls connection state, and shows const entry = await upsertChannelEntry(id, { name: id.toLowerCase(), type: "channel", isDM: false, platform: "slack" }); await saveChannelMeta(entry.slug, { ...defaultChannelMeta({ channelId: id, name: id.toLowerCase(), type: "channel", isDM: false }), allowNetwork: true }); } - statuses.set("C_VPN_SETUP", { ...off, configured: false, state: "unconfigured", message: "No VPN configured." }); + statuses.set("C_VPN_SETUP", { ...off, configured: false, state: "unconfigured", message: "VPN is not configured. An administrator must import the profile and prepare the service first." }); const browser = await chromium.launch({ headless: true, args: ["--no-sandbox"] }); t.after(() => browser.close()); const context = await browser.newContext(); @@ -161,6 +161,10 @@ test("browser VPN switch applies immediately, polls connection state, and shows await page.goto(`${base}/conversations/channel/C_VPN_SETUP`); await page.waitForFunction(() => globalThis.document.querySelector("#channel-detail .ch-vpn-state")?.textContent.includes("Not configured")); assert.equal(await page.locator("#channel-detail .ch-vpn-enabled").isDisabled(), true); - assert.match(await page.locator("#channel-detail .ch-vpn-state").textContent(), /administrator must import/); + const setupText = await page.locator("#channel-detail .ch-vpn-state").textContent(); + assert.match(setupText, /administrator must import/); + // QA-0925: the server's own "VPN is not configured. …" used to be appended to the hint below, + // saying the same thing twice. + assert.equal(setupText.includes("VPN is not configured"), false, "an unconfigured VPN is explained once"); assert.deepEqual(errors, []); }); From c83eddf8e32601f363e1f02b93d12a3ac726a71a Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 13:41:10 +0300 Subject: [PATCH 08/14] fix: tell every Composio run how a file reaches a Composio tool The staging route lived only in the gateway-usage guide's sharing page. A Codex turn that read the guide's front page but not that page (QA-0925 FSHARE-02) base64'd a PDF through the Composio workbench instead of calling stage_file_for_composio, and described it as staging. Every per-run Composio identity line now carries the handoff rule: stage with the identity that runs the destination tool, use an upload link for URL-only tools, and never push file bytes through the workbench to get around staging. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- CHANGELOG.md | 3 +++ FEATURES.md | 7 +++++-- TEST-PLAN.md | 6 ++++++ src/gateway/mcp.js | 10 +++++++--- test/composio-identity-preamble.test.js | 13 +++++++++++++ 5 files changed, 34 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 165452a..098c827 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,9 @@ product overview. ## Unreleased +- Handing a generated file to Drive, Gmail or another Composio tool now goes through the gateway's + staging on every engine: the rule rides each run's Composio identity notice, so Codex no longer + falls back to pushing the file through Composio's workbench as base64. - Slack **⚙️ Settings** no longer claims a channel pinned to Claude inherits the Codex default model (or the reverse): an unset model or effort now names what the channel's own harness would actually use. The web admin's VPN status also stops saying "not configured" twice. diff --git a/FEATURES.md b/FEATURES.md index 34f8cb9..0e1182a 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -1809,8 +1809,11 @@ A categorized catalog of what's shipped. Cross-linked to `TEST-PLAN.md` checks. object to pass straight through. The caller names which identity will run the destination tool (`user` → `composio-user`, `agent` → `composio-agent`) and the key resolved for THAT identity is the one spent, through the same precedence the MCP config uses; a named identity with no key is - reported rather than silently replaced by the other one. The key never enters the container, - never reaches the model, and never appears in an error message. Nothing is published. + reported rather than silently replaced by the other one. Every run that has a Composio identity + is also told this handoff rule in its per-run identity line (not only in the gateway-usage guide), + so an engine that never opens the guide's sharing page still stages instead of relaying base64. + The key never enters the container, never reaches the model, and never appears in an error + message. Nothing is published. The REST upload accepts only a Composio PROJECT API key. The keys this gateway stores in personal mode are CONSUMER keys (`ck_…`, the hosted MCP's credential), which that endpoint rejects, so with a consumer key the gateway stages through the hosted MCP's own workbench instead: the daemon diff --git a/TEST-PLAN.md b/TEST-PLAN.md index 2592a53..8b2cfce 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -85,6 +85,12 @@ and require the file to open in Drive with the right bytes. Repeat with "my Drive" vs "your Drive" and confirm the staged identity matches the one the upload ran as. Record the `composio_file_staged` audit event. +- [x] Automated: every per-run Composio identity line (both, user-only, agent-only) carries the + handoff rule — `stage_file_for_composio` with the identity that runs the destination tool, a + `create_public_file_link` upload link for URL-only tools, and never file bytes as base64 or + chunks through the workbench to get around staging — and an identity-less run gets no line + (`test/composio-identity-preamble.test.js`). QA-0925 FSHARE-02: a Codex turn that read the + guide's front page but not its sharing page base64'd a PDF through the workbench. ## Temporary public file links diff --git a/src/gateway/mcp.js b/src/gateway/mcp.js index bf6da2d..eceb13a 100644 --- a/src/gateway/mcp.js +++ b/src/gateway/mcp.js @@ -95,17 +95,21 @@ export function composioIdentitiesForRun({ clean = false, principalTrusted = tru // content-free: server names and roles only, never a token, an address or an account label. export function composioIdentityPreamble({ user = false, agent = false } = {}) { const ownership = " These logical identities do not establish the connected service owner; discover account metadata through the selected identity before claiming ownership. The agent identity is not necessarily shared across channels. Matching service owners never authorize substituting identities."; + // QA-0925 (FSHARE-02): the routing lived only in the gateway-usage guide, and a Codex turn that + // skipped that page pushed a PDF through the workbench as base64 instead. Every run that has a + // Composio identity needs the one handoff rule, whichever engine and whichever pages it reads. + const files = " To hand a file from this folder to a Composio tool that takes a file object (Drive upload, email attachment, …), call `gateway` → `stage_file_for_composio` with `identity` set to the identity that will run that tool and pass its result on unchanged; a tool that only ingests by URL takes a `create_public_file_link` upload link instead. Never push a file's bytes as base64 or chunks through the workbench or another Composio tool to get around staging, including when staging fails or the file is too large."; if (user && agent) { return "[Composio identities in THIS run: `composio-user` (the requester's own accounts) and `composio-agent` (the shared agent's own). " + - "Reads and searches may use either or both identities without asking which account unless the user restricts the account or scope. Writes, sends and other state changes require the intended identity and connected account: reuse an established choice, or ask \"which account?\" if unresolved before mutating; continue independent authorized reads." + ownership + "]\n\n"; + "Reads and searches may use either or both identities without asking which account unless the user restricts the account or scope. Writes, sends and other state changes require the intended identity and connected account: reuse an established choice, or ask \"which account?\" if unresolved before mutating; continue independent authorized reads." + ownership + files + "]\n\n"; } if (user) { return "[Composio identities in THIS run: `composio-user` only (the requester's own accounts). " + - "There is no shared agent identity here, so a request for the agent's own accounts (\"your inbox\") has nothing to read — say so and stop." + ownership + "]\n\n"; + "There is no shared agent identity here, so a request for the agent's own accounts (\"your inbox\") has nothing to read — say so and stop." + ownership + files + "]\n\n"; } if (agent) { return "[Composio identities in THIS run: `composio-agent` only (the shared agent's connections). " + - "A request phrased for the person asking (\"my inbox\", \"my calendar\") cannot be served here: say so and stop, do not read `composio-agent` to answer it." + ownership + "]\n\n"; + "A request phrased for the person asking (\"my inbox\", \"my calendar\") cannot be served here: say so and stop, do not read `composio-agent` to answer it." + ownership + files + "]\n\n"; } return ""; } diff --git a/test/composio-identity-preamble.test.js b/test/composio-identity-preamble.test.js index 878faed..6e47f8a 100644 --- a/test/composio-identity-preamble.test.js +++ b/test/composio-identity-preamble.test.js @@ -58,6 +58,19 @@ test("the run's identity set mirrors the servers the MCP config would carry", () ); }); +test("every identity variant routes a file handoff through the gateway's staging tool", () => { + // QA-0925 FSHARE-02: a Codex turn that never opened the guide's sharing page base64'd a PDF + // through the Composio workbench. The rule has to ride the per-run line every engine reads. + for (const ids of [{ user: true, agent: true }, { user: true }, { agent: true }]) { + const line = composioIdentityPreamble(ids); + assert.match(line, /`gateway` → `stage_file_for_composio` with `identity` set to the identity that will run that tool/); + assert.match(line, /only ingests by URL takes a `create_public_file_link` upload link instead/); + assert.match(line, /Never push a file's bytes as base64 or chunks through the workbench or another Composio tool to get around staging/); + assert.equal(line.trim().split("\n").length, 1, "still one line"); + } + assert.equal(composioIdentityPreamble({}), "", "no identity, no rule"); +}); + test("the identity line names only servers and roles — never a token, an address or an account", () => { const both = composioIdentityPreamble({ user: true, agent: true }); assert.match(both, /^\[Composio identities in THIS run: `composio-user`.*`composio-agent`/); From dbdba975ae90b4796bd1788422d969493b6b23d0 Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 13:58:59 +0300 Subject: [PATCH 09/14] fix: resync a Drive folder whose sync record holds no files rclone's --resync of two empty sides succeeds but leaves listings that record no file, and every later bisync then aborts with exit 7 ("Empty prior Path1 listing ... Must run --resync to recover"). The gateway had already written its resync sentinel, so a channel linked to an empty Drive folder synced once and then failed every tick (QA-0925 GDS-02). A pass now resyncs again when both prior listings exist and record no file, or - for channels already wedged - when both were set aside as header-only .lst-err. Any other missing listing stays an error: after a deliberate delete-everything rclone sets aside NON-empty listings, and a resync there would copy the deleted files back. A failed forced resync keeps its state instead of dropping it. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- CHANGELOG.md | 3 ++ FEATURES.md | 8 ++++- TEST-PLAN.md | 12 +++++++ src/gateway/drivesync.js | 36 +++++++++++++++++-- test/drivesync-manual.test.js | 8 +++-- test/drivesync.test.js | 66 ++++++++++++++++++++++++++++++++++- 6 files changed, 126 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 165452a..506d903 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,9 @@ product overview. ## Unreleased +- Google Drive sync no longer stalls on a folder that was empty when it was linked. The first sync + of two empty sides succeeded, but every later sync then failed (rclone: "Empty prior Path1 + listing"), so files never moved. A sync record that holds no files now triggers a fresh resync. - Slack **⚙️ Settings** no longer claims a channel pinned to Claude inherits the Codex default model (or the reverse): an unset model or effort now names what the channel's own harness would actually use. The web admin's VPN status also stops saying "not configured" twice. diff --git a/FEATURES.md b/FEATURES.md index 34f8cb9..c677da1 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -3018,7 +3018,13 @@ are retired, bullet by bullet; everything else stands. through daemon IPC (`drivesync` kind) so the pass runs in the daemon on both MCP transports, shares the per-channel in-flight guard and outlives the turn; it waits ~40 s for the outcome and otherwise says the pass is still running. `get_channel_drive_folder` now also reports the last - pass (time, ok/failed, first-resync, reason). Manual passes obey the same global switch, key and + pass (time, ok/failed, first-resync, reason). A pass whose prior rclone listings record no file + (including the empty listings rclone set aside as `.lst-err` after refusing them) runs as a + `--resync` again, so a folder linked while empty on both sides starts syncing as soon as either + side gets a file instead of failing every tick with rclone's exit 7. Any other missing listing + stays an error: after a deliberate delete-everything rclone sets aside NON-empty listings, and a + resync there would copy the deleted files back. + Manual passes obey the same global switch, key and rclone checks as the schedule; a pass already running for a channel is never doubled, and a manual sweep never stacks on the scheduled one. Status surfaces show a concise diagnostic, never the raw rclone tail. Admin API: `POST /api/channels/:id/sync-now`, `GET diff --git a/TEST-PLAN.md b/TEST-PLAN.md index 2592a53..a8c2009 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -3280,6 +3280,18 @@ structural invariants are automated; rendered navigation and feature claims also `MEMORY.md`, `memory/`, `uploads/` are never pushed to Drive nor overwritten from it. - [ ] A failed first run leaves no half-baked bisync state (the state dir is dropped, so the next tick retries with `--resync`). +- [x] A prior listing that records no file (`.lst`, or the `.lst-err` rclone set aside after refusing + it) forces `--resync` again, and any other missing listing stays an error + (`test/drivesync.test.js`, with real-rclone cases that skip when rclone is absent: an empty + folder that later gets a file syncs; deleting every file on one side is NOT undone by a + forced resync; a failed forced resync keeps its state). rclone's `--resync` of two EMPTY sides + succeeds but leaves an empty listing, and every later pass then aborted with exit 7 "Empty + prior Path1 listing … Must run --resync to recover". QA-0925: a channel linked to an empty + shared-drive folder synced "OK" once and then failed every tick once a file appeared. +- [ ] Live (engine-independent): link a channel to an EMPTY Drive folder with an empty local + `Drive/`, click **Sync now** (ok), then put a file in the local `Drive/` and click **Sync now** + again. Pass: the second pass succeeds and the file appears in the Drive folder; a file added + in Drive then reaches `Drive/` on the next pass; no `drivesync_error` event. - [x] Set the Drive folder link via the gateway MCP tools (`src/mcp/gateway-server.js`): `set_channel_drive_folder`/`clear_channel_drive_folder` are admin-gated (`requireAdmin`) and reuse the shared `parseDriveFolderId` (junk link → rejected before any write) + `testChannelSync` diff --git a/src/gateway/drivesync.js b/src/gateway/drivesync.js index 3843be4..219219e 100644 --- a/src/gateway/drivesync.js +++ b/src/gateway/drivesync.js @@ -21,7 +21,7 @@ import path from "node:path"; import { spawn, spawnSync } from "node:child_process"; -import { existsSync, mkdirSync, rmSync, readFileSync, writeFileSync, chmodSync } from "node:fs"; +import { existsSync, mkdirSync, rmSync, readFileSync, readdirSync, writeFileSync, chmodSync } from "node:fs"; import { getDriveSyncEnabled, getDriveSyncKeyFile, @@ -153,6 +153,33 @@ export function needsResync(stateDir) { return !existsSync(resyncSentinel(stateDir)); } +// rclone refuses to bisync against an EMPTY prior listing — exit 7, "Empty prior Path1 listing. +// Cannot sync to an empty directory … Must run --resync to recover" — and a --resync of two empty +// sides succeeds while leaving exactly such a listing behind (rclone then renames it to .lst-err on +// the failed pass). So a channel linked to an empty Drive folder before anyone put a file on either +// side synced "successfully" once and then failed every tick forever (QA-0925). A listing that +// records NO file has no deletion a fresh --resync could miss, so that — and only that — resyncs +// again. A listing that is missing for any other reason stays an error: after a deliberate +// delete-everything, rclone aborts and renames NON-empty listings to .lst-err, and resyncing there +// would silently copy every deleted file back. +function listingEntries(stateDir, names, suffix) { + const name = names.find((entry) => entry.endsWith(suffix)); + if (!name) return null; + try { + return readFileSync(path.join(stateDir, name), "utf8").split("\n").filter((line) => line.trim() && !line.startsWith("#")).length; + } catch { return null; } +} +export function priorListingEmpty(stateDir) { + let names; + try { names = readdirSync(stateDir); } catch { return false; } + const sides = [".path1", ".path2"]; + const live = sides.map((side) => listingEntries(stateDir, names, `${side}.lst`)); + if (live.every((n) => n !== null)) return live.every((n) => n === 0); + if (live.some((n) => n !== null)) return false; // one side missing: not the empty-resync shape + const aborted = sides.map((side) => listingEntries(stateDir, names, `${side}.lst-err`)); + return aborted.every((n) => n === 0); +} + // Resolve the effective service-account key FILE for rclone. A pasted JSON key (write-only setting) // wins: it's materialized to a chmod-600 file in the runtime config dir — outside every channel // sandbox, and passed to rclone as a PATH so the private key never enters argv or the child env. @@ -245,7 +272,8 @@ async function syncOne({ slug, channelId, folderId, meta }, { bin, keyFile, subj const workDir = effectiveWorkDir(slug, meta || {}); // the channel's real folder (honors a custom workDir) const localPath = syncSubdir(workDir); const stateDir = channelWorkDir(slug); - const firstRun = needsResync(stateDir); + const initial = needsResync(stateDir); + const firstRun = initial || priorListingEmpty(stateDir); mkdirSync(localPath, { recursive: true }); // both sides must exist before bisync mkdirSync(stateDir, { recursive: true }); const args = buildBisyncArgs({ localPath, folderId, keyFile, subject, workDir: stateDir, conflict, firstRun }); @@ -268,7 +296,9 @@ async function syncOne({ slug, channelId, folderId, meta }, { bin, keyFile, subj // A failed first run must retry --resync next tick against a clean slate, so drop the // (now-stale) listing state. No sentinel was written, so the retry stays a first run either // way — this only removes a half-built baseline rclone would otherwise read. - if (firstRun) { try { rmSync(stateDir, { recursive: true, force: true }); } catch {} } + // Only a genuine first run may be dropped: a failed forced resync keeps the sentinel and the + // listings that show what happened. + if (initial) { try { rmSync(stateDir, { recursive: true, force: true }); } catch {} } await logEvent("drivesync_error", { slug, channel: channelId, trigger, code: res.code, signal: res.signal, outcome: res.outcome?.kind, tail: res.tail.slice(-800) }); const detail = conciseProcessDiagnostic(res.tail, 300); console.error(`[drivesync] ${slug} bisync ${res.outcome?.summary || "failed before it completed"}${detail ? `: ${detail}` : ""}`); diff --git a/test/drivesync-manual.test.js b/test/drivesync-manual.test.js index 25b0787..e5b93cb 100644 --- a/test/drivesync-manual.test.js +++ b/test/drivesync-manual.test.js @@ -22,8 +22,10 @@ const SA_JSON = JSON.stringify({ }); // A fake rclone: `version` succeeds; a pass waits while `block` exists, records its argv in -// `calls`, and fails with a diagnostic while `fail` exists. Paths are baked in because the child -// env is the gateway's curated one, not this test's. +// `calls`, and fails with a diagnostic while `fail` exists. A successful pass leaves a one-file +// listing in its --workdir, as a real bisync over a non-empty folder does (an EMPTY listing forces +// the next pass to --resync, drivesync.test.js). Paths are baked in because the child env is the +// gateway's curated one, not this test's. const fake = tempDir("cg-fake-rclone-"); const bin = path.join(fake, "rclone"); const calls = path.join(fake, "calls"); @@ -34,6 +36,8 @@ writeFileSync(bin, `#!/bin/sh while [ -f "${block}" ]; do sleep 0.05; done echo "$*" >> "${calls}" if [ -f "${fail}" ]; then echo "ERROR : Failed to bisync: googleapi: Error 403: insufficient permissions" >&2; exit 2; fi +wd=""; prev=""; for a in "$@"; do [ "$prev" = "--workdir" ] && wd="$a"; prev="$a"; done +if [ -n "$wd" ]; then for side in path1 path2; do printf '# bisync listing v1\\n- 2 - - 2026-09-25T00:00:00Z "f.txt"\\n' > "$wd/fake.$side.lst"; done; fi exit 0 `); chmodSync(bin, 0o755); diff --git a/test/drivesync.test.js b/test/drivesync.test.js index 6f6fc69..79bcca8 100644 --- a/test/drivesync.test.js +++ b/test/drivesync.test.js @@ -11,7 +11,7 @@ import { ensureTestEnv, tempDir } from "./helpers.js"; ensureTestEnv(); -const { parseDriveFolderId, syncSubdir, buildBisyncArgs, buildTestArgs, selectSyncChannels, isServiceAccountJson, resolveDriveSyncKeyFile, driveSyncResultOutput, needsResync, resyncSentinel, rcloneAvailable } = await import("../src/gateway/drivesync.js"); +const { parseDriveFolderId, syncSubdir, buildBisyncArgs, buildTestArgs, selectSyncChannels, isServiceAccountJson, resolveDriveSyncKeyFile, driveSyncResultOutput, needsResync, resyncSentinel, priorListingEmpty, rcloneAvailable } = await import("../src/gateway/drivesync.js"); const { saveSettings } = await import("../src/config/settings.js"); const { configDir } = await import("../src/config/paths.js"); @@ -171,6 +171,70 @@ test("a crash mid-first-sync no longer wedges a channel into permanent non-resyn assert.ok(!argsFor(crashed).includes("--resync")); }); +test("only a listing that records no file forces another --resync; any other missing listing stays an error", () => { + // QA-0925: a channel linked to an empty Drive folder resynced two empty sides, then every later + // tick failed "Empty prior Path1 listing … Must run --resync to recover" — forever. + const dir = tempDir("drivesync-listing"); + assert.equal(priorListingEmpty(path.join(dir, "absent")), false, "no state dir: the sentinel decides, not this"); + assert.equal(priorListingEmpty(dir), false, "no listing files at all stays an error"); + const p1 = path.join(dir, "local_Drive.._drive_.path1.lst"); + const p2 = path.join(dir, "local_Drive.._drive_.path2.lst"); + writeFileSync(p1, "# bisync listing v1 from 2026-09-25T10:21:21Z\n"); + assert.equal(priorListingEmpty(dir), false, "one side missing is not the empty-resync shape"); + writeFileSync(p2, "# bisync listing v1 from 2026-09-25T10:21:21Z\n"); + assert.equal(priorListingEmpty(dir), true, "two header-only listings record no file"); + writeFileSync(p1, '# bisync listing v1\n- 2 - - 2026-09-25T10:41:39Z "s.txt"\n'); + assert.equal(priorListingEmpty(dir), false, "a listing that records a file is never resynced over"); + // A channel ALREADY wedged in production: rclone renamed the empty listings to .lst-err. + const wedged = tempDir("drivesync-wedged"); + writeFileSync(path.join(wedged, "x.path1.lst-err"), "# header\n"); + writeFileSync(path.join(wedged, "x.path2.lst-err"), "# header\n"); + assert.equal(priorListingEmpty(wedged), true, "header-only .lst-err on both sides recovers"); + // A deliberate delete-everything: rclone aborts and renames NON-empty listings to .lst-err. + // Resyncing here would copy every deleted file back, so it must stay an error. + const deleted = tempDir("drivesync-deleted"); + writeFileSync(path.join(deleted, "x.path1.lst-err"), '# header\n- 2 - - t "a.txt"\n'); + writeFileSync(path.join(deleted, "x.path2.lst-err"), '# header\n- 2 - - t "a.txt"\n'); + assert.equal(priorListingEmpty(deleted), false); + const partial = tempDir("drivesync-partial"); + writeFileSync(path.join(partial, "x.path2.lst-new"), '- 2 - - t "a"\n'); + assert.equal(priorListingEmpty(partial), false, "a crashed pass's .lst-new leftovers never trigger a resync"); + const source = readFileSync(new URL("../src/gateway/drivesync.js", import.meta.url), "utf8"); + assert.match(source, /const firstRun = initial \|\| priorListingEmpty\(stateDir\);/); + assert.match(source, /if \(initial\) \{ try \{ rmSync\(stateDir/, "a failed FORCED resync keeps its state"); +}); + +test("real rclone: two empty sides, then a new file, syncs once the empty listing forces --resync", { skip: !rcloneAvailable("rclone") && "rclone not installed" }, async () => { + const { spawnSync } = await import("node:child_process"); + const root = tempDir("drivesync-rclone"); + const a = path.join(root, "a"), b = path.join(root, "b"), w = path.join(root, "w"); + for (const d of [a, b, w]) mkdirSync(d, { recursive: true }); + const bisync = (resync) => spawnSync("rclone", ["bisync", a, b, "--workdir", w, "--create-empty-src-dirs", ...(resync ? ["--resync", "--resync-mode", "newer"] : []), "-q"], { encoding: "utf8" }); + assert.equal(bisync(true).status, 0, "first --resync of two empty sides succeeds"); + assert.equal(priorListingEmpty(w), true, "…and leaves an empty prior listing behind"); + writeFileSync(path.join(a, "x.txt"), "hi\n"); + assert.equal(bisync(false).status, 7, "the unpatched steady-state pass is rclone's critical exit 7"); + const fixed = bisync(priorListingEmpty(w)); + assert.equal(fixed.status, 0, fixed.stderr); + assert.equal(readFileSync(path.join(b, "x.txt"), "utf8"), "hi\n", "the new file reached the other side"); + assert.equal(priorListingEmpty(w), false, "and the next pass is an ordinary bisync"); +}); + +test("real rclone: deleting every file on one side is never undone by a forced resync", { skip: !rcloneAvailable("rclone") && "rclone not installed" }, async () => { + const { spawnSync } = await import("node:child_process"); + const root = tempDir("drivesync-rclone-delete"); + const a = path.join(root, "a"), b = path.join(root, "b"), w = path.join(root, "w"); + for (const d of [a, b, w]) mkdirSync(d, { recursive: true }); + for (const n of ["1", "2", "3"]) writeFileSync(path.join(a, `${n}.txt`), n); + const bisync = (resync) => spawnSync("rclone", ["bisync", a, b, "--workdir", w, ...(resync ? ["--resync", "--resync-mode", "newer"] : []), "-q"], { encoding: "utf8" }); + assert.equal(bisync(true).status, 0); + assert.equal(bisync(false).status, 0, "a normal pass after the baseline"); + for (const n of ["1", "2", "3"]) rmSync(path.join(a, `${n}.txt`)); + assert.notEqual(bisync(false).status, 0, "rclone refuses to empty a side"); + assert.equal(priorListingEmpty(w), false, "so the gateway must NOT resync (that would restore the files)"); + assert.equal(existsSync(path.join(a, "1.txt")), false, "the deletion still stands"); +}); + test("the resync sentinel is written only on a successful pass", () => { const source = readFileSync(new URL("../src/gateway/drivesync.js", import.meta.url), "utf8"); const success = source.indexOf("if (res.ok) {"); From be4da55ae3a3087489e508a9576687324415198e Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 14:11:30 +0300 Subject: [PATCH 10/14] fix: open VS Code over SSH in the channel folder, not the container home VS Code Remote-SSH opened an empty window in /home/agent, and its Open Folder dialog started there, so a developer had to climb the tree to reach the channel folder where CLAUDE.md, the skills and the sessions live (QA-0925, reported by Tiberiu). Each SSH session now seeds files.dialog.defaultPath in the container's VS Code machine settings with the channel's effective work folder (custom folders included). The merge keeps VS Code's own keys, follows a changed folder through a sidecar, never overrides a value the developer set, leaves non-JSON files alone, and only warns on failure. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- CHANGELOG.md | 2 ++ FEATURES.md | 6 +++++- TEST-PLAN.md | 8 ++++++++ docs/SSH-ACCESS.md | 7 +++++-- src/gateway/ssh-session.js | 41 +++++++++++++++++++++++++++++++++++++ src/mcp/tools/ssh-access.js | 2 +- test/ssh-session.test.js | 41 +++++++++++++++++++++++++++++++++++++ 7 files changed, 103 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fa70de8..8af5826 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,8 @@ product overview. ## Unreleased +- VS Code over SSH now starts in the channel's folder: File → Open Folder in a Remote-SSH window + opens at the channel folder (or its custom folder) instead of the container's home. - Handing a generated file to Drive, Gmail or another Composio tool now goes through the gateway's staging on every engine: the rule rides each run's Composio identity notice, so Codex no longer falls back to pushing the file through Composio's workbench as base64. diff --git a/FEATURES.md b/FEATURES.md index 8aacd2b..929be08 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -2405,7 +2405,11 @@ are retired, bullet by bullet; everything else stands. container's namespaces; no container and no extra host port ever listens; the channel is named in the ProxyCommand, so one key reaches several channels at once. A container with a live session is never idle-stopped or evicted, and a rebuild waits for it like for a run; a dead peer - is reaped by `ClientAlive` in about three minutes. Sessions are `ssh_sessions` rows and + is reaped by `ClientAlive` in about three minutes. Each session also seeds VS Code's remote + machine setting `files.dialog.defaultPath` with the channel's effective work folder (a custom + folder included), so Remote-SSH's File → Open Folder starts there instead of `/home/agent`; + the merge is best effort, follows a changed folder, and never overrides a developer's own value. + Sessions are `ssh_sessions` rows and `ssh_session_start`/`ssh_session_end` events; refusals are `ssh_attach_refused` with the reason the developer saw. The image ships `openssh-server` (spec 1.4.0) and marks the `agent` account key-only (`*`, not useradd's locked `!`). It carries no SSH host private key: the package's diff --git a/TEST-PLAN.md b/TEST-PLAN.md index efa6bec..5733964 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -5425,6 +5425,14 @@ are the v0.8 production deployment gate and are executed in the QA loop that fol - [x] Unit: the image ships `cg-sshd` (POSIX sh clean) and the spec is 1.4.0 in both `containers/versions.json` and `image-paths.js` (automated: `test/container-image.test.js`, `test/container-durability.test.js`). +- [x] Automated (`test/ssh-session.test.js`): session prep runs the VS Code start-folder seed in the + channel's container with the effective work folder; the seed creates the machine settings, + keeps VS Code's own keys, follows a changed work folder, never overrides a developer-set + `files.dialog.defaultPath`, never rewrites a non-JSON file, and a failed seed never blocks the + session. QA-0925: Remote-SSH landed in `/home/agent`. +- [ ] LIVE (engine-independent): connect with VS Code Remote-SSH from its own menu to a channel with + a custom work folder. Pass: File → Open Folder opens at that folder (not `/home/agent`), OK + opens it, and a new terminal's `pwd` is that folder. - [ ] LIVE (engine-independent, Airtable CTR-31): on the gateway host run `npm run build:image`, then `sudo CG_SSH_HOST= bash scripts/install-ssh-access.sh`; within a minute the daemon log shows `[ssh] attach socket`. As Apps, in `cg-testing-claude-bash`, send "add my SSH key `. +To open VS Code directly on the channel folder, use the command "show SSH access" prints: +`code --remote ssh-remote+acme-app `. Connecting from Remote-SSH's own menu opens an +empty window instead; its File → Open Folder dialog starts in the channel folder (a custom work +folder included), because every session seeds `files.dialog.defaultPath` in the container's VS Code +machine settings — merge-only, and never over a value you set yourself. Inside, you are user `agent` in the channel's work folder (an interactive login starts there; image spec 1.5.1), with the same environment an engine turn gets and the channel's persistent diff --git a/src/gateway/ssh-session.js b/src/gateway/ssh-session.js index 84c0ff2..ff0c9b7 100644 --- a/src/gateway/ssh-session.js +++ b/src/gateway/ssh-session.js @@ -129,6 +129,36 @@ export function renderAccessOnlyCredentials(relay) { }, null, 2); } +// VS Code Remote-SSH opens an empty window in the container's HOME, and its Open Folder dialog and +// terminal start there, so a developer had to climb out of /home/agent to reach the channel folder +// (QA-0925). `files.dialog.defaultPath` in the REMOTE machine settings is what that dialog starts in +// when the window has no recent folder. Merge-only: the file is the channel's own VS Code state +// (VS Code writes to it too), a sidecar remembers the value WE wrote so a changed work folder moves +// it, and a value the developer set themselves — or a file that is not plain JSON — is left alone. +// argv[1] = settings file, argv[2] = sidecar, argv[3] = the channel work folder. +export const VSCODE_MACHINE_SETTINGS = "/home/agent/.vscode-server/data/Machine/settings.json"; +export const VSCODE_FOLDER_SEED = ` +const fs = require("node:fs"); +const path = require("node:path"); +const [file, sidecar, folder] = process.argv.slice(1); +const key = "files.dialog.defaultPath"; +let config = {}; +try { config = JSON.parse(fs.readFileSync(file, "utf8")); } +catch (error) { if (error.code !== "ENOENT") process.exit(0); } +if (!config || typeof config !== "object" || Array.isArray(config)) process.exit(0); +let ours = ""; +try { ours = fs.readFileSync(sidecar, "utf8").trim(); } catch {} +const current = config[key]; +if (current !== undefined && current !== ours) process.exit(0); +if (current === folder && ours === folder) process.exit(0); +config[key] = folder; +fs.mkdirSync(path.dirname(file), { recursive: true, mode: 0o700 }); +const temporary = file + ".cg-" + process.pid; +fs.writeFileSync(temporary, JSON.stringify(config, null, "\\t") + "\\n", { mode: 0o600 }); +fs.renameSync(temporary, file); +fs.writeFileSync(sidecar, folder + "\\n", { mode: 0o600 }); +`; + /** * Prepare (or refresh) one developer's session in one channel. Every step past the relay is best * effort and REPORTED, never fatal: a session with plain Claude beats no session, and the status @@ -181,6 +211,17 @@ export async function prepareSshSession({ target, entry, meta = {}, user, cliBin } } + // 2b. VS Code lands in the channel folder, not the container HOME. Best effort: a failure only + // costs the developer a few clicks, so it is logged and never blocks the session. + if (target.workDir) { + try { + await exec(["exec", target.container.name, "node", "-e", VSCODE_FOLDER_SEED, + VSCODE_MACHINE_SETTINGS, `${VSCODE_MACHINE_SETTINGS}.cg-default-folder`, String(target.workDir)]); + } catch (error) { + log?.warn?.(`[ssh] ${slug}/${user.id}: VS Code start folder not set (${String(error?.message || error).slice(0, 120)})`); + } + } + // 3. The turn-equivalent files: lockdown, MCP payload, run environment. const clean = Boolean(meta.cleanMode); const adapter = requireAdapter("claude"); diff --git a/src/mcp/tools/ssh-access.js b/src/mcp/tools/ssh-access.js index 05d5dbd..51de494 100644 --- a/src/mcp/tools/ssh-access.js +++ b/src/mcp/tools/ssh-access.js @@ -182,7 +182,7 @@ export function register(server, ctx) { else if (!mine && createdBy) lines.push(`• You are not granted here${myKeys.length ? "" : " and have no key registered"}.`); if (setup.configured) { lines.push("", "Once granted, add this to `~/.ssh/config` on your laptop (your usual key; nothing per channel), then `ssh " + slug + "` or open it with VS Code Remote-SSH:", "```", connectSnippet({ endpoint: setup.endpoint, channel: slug }), "```", - `To open VS Code straight on the channel folder: \`code --remote ssh-remote+${slug} ${effectiveWorkDir(slug, meta)}\` (the Open Folder dialog otherwise starts in /home/agent).`, + `To open VS Code straight on the channel folder: \`code --remote ssh-remote+${slug} ${effectiveWorkDir(slug, meta)}\` (from Remote-SSH's own connect menu, File → Open Folder starts in that folder; press OK).`, "Inside you are user `agent` in the channel's work folder with its `/home/agent`, CLI logins and Codex. `claude` there is this channel's Claude exactly as a message here gets it: the channel's tool policy, the gateway tools (no background jobs or approval cards — no thread to post into), your own Composio accounts as `composio-user`, the channel's as `composio-agent`, the channel's MCP servers and its secrets by name — signed in as the gateway's own account, whose usage it counts against. Everyone in the box shares that one user; set your git identity per session. A daemon restart drops sessions — just reconnect."); } return text(lines.join("\n")); diff --git a/test/ssh-session.test.js b/test/ssh-session.test.js index 3de8997..3278382 100644 --- a/test/ssh-session.test.js +++ b/test/ssh-session.test.js @@ -222,3 +222,44 @@ test("the ssh toolset is the control plane minus the thread-bound tools; ssh_ses assert.ok(RUN_ORIGINS.includes(session.SSH_SESSION_ORIGIN)); assert.equal(PRINCIPAL_KIND_BY_ORIGIN[session.SSH_SESSION_ORIGIN], "user"); }); + +test("VS Code over SSH starts its Open Folder dialog in the channel folder, never over the developer's own choice", () => { + // QA-0925: Remote-SSH landed in /home/agent and the developer had to climb to the channel folder. + const dir = mkdtempSync(path.join(scratch, "vscode-seed-")); + const file = path.join(dir, "Machine", "settings.json"); + const sidecar = `${file}.cg-default-folder`; + const seed = (folder) => execFileSync(process.execPath, ["-e", session.VSCODE_FOLDER_SEED, file, sidecar, folder]); + const read = () => JSON.parse(readFileSync(file, "utf8")); + seed("/home/management/ChannelGate/slack/a"); + assert.equal(read()["files.dialog.defaultPath"], "/home/management/ChannelGate/slack/a", "absent file: created with the folder"); + // VS Code's own entries survive the merge. + writeFileSync(file, JSON.stringify({ ...read(), "github.copilot.chat.codeGeneration.instructions": [{ text: "x" }] })); + seed("/home/management/custom-folder"); + assert.equal(read()["files.dialog.defaultPath"], "/home/management/custom-folder", "our value follows a changed work folder"); + assert.deepEqual(read()["github.copilot.chat.codeGeneration.instructions"], [{ text: "x" }]); + // The developer points it somewhere else: left alone from then on. + writeFileSync(file, JSON.stringify({ ...read(), "files.dialog.defaultPath": "/home/agent/projects" })); + seed("/home/management/ChannelGate/slack/a"); + assert.equal(read()["files.dialog.defaultPath"], "/home/agent/projects"); + // A file that is not plain JSON (comments) is never rewritten. + writeFileSync(file, "// mine\n{}\n"); + seed("/home/management/ChannelGate/slack/a"); + assert.equal(readFileSync(file, "utf8"), "// mine\n{}\n"); +}); + +test("session prep seeds the VS Code start folder with the channel's effective work folder, and a failure never blocks the session", async () => { + const t = target("ssh-vscode"); + const { execs, deps } = fakes(); + await session.prepareSshSession({ target: t, entry: { slug: "ssh-vscode", channelId: "C_VS" }, meta: {}, user, cliBin: "podman", log: { warn() {} } }, deps); + const call = execs.find((e) => e.args.includes(session.VSCODE_FOLDER_SEED)); + assert.ok(call, "the seed ran"); + assert.deepEqual(call.args.slice(-3), [session.VSCODE_MACHINE_SETTINGS, `${session.VSCODE_MACHINE_SETTINGS}.cg-default-folder`, "/work/ssh-vscode"]); + assert.equal(call.args[1], "cg-ssh-vscode", "inside the channel's own container"); + const warnings = []; + const failing = fakes(); + const runCommand = failing.deps.runCommand; + failing.deps.runCommand = async (bin, args, options) => { if (args.includes(session.VSCODE_FOLDER_SEED)) throw new Error("exec failed"); return runCommand(bin, args, options); }; + const result = await session.prepareSshSession({ target: target("ssh-vscode-fail"), entry: { slug: "ssh-vscode-fail", channelId: "C_VF" }, meta: {}, user, cliBin: "podman", log: { warn: (m) => warnings.push(m) } }, failing.deps); + assert.deepEqual(result.problems, [], "the session is still fully prepared"); + assert.ok(warnings.some((m) => m.includes("VS Code start folder not set"))); +}); From 88b005b568c43707cd1ee6e9b4ff2a3c2af8341d Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 14:38:06 +0300 Subject: [PATCH 11/14] feat: show Slack Settings pages as a row of tabs The five pages were picked from a Page dropdown, which took two clicks and hid which pages exist. They are now one row of tab buttons at the top - General, Resume, MCP, Skills, Secrets - with the open page highlighted. The short names keep the row on one line, which is what the dropdown had been introduced to fix. Each tab carries the same "tab" command the dropdown did, so a Settings view opened while the dropdown shipped still navigates. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- CHANGELOG.md | 2 ++ FEATURES.md | 7 +++-- TEST-PLAN.md | 14 ++++++---- src/slack/channel-settings.js | 43 +++++++++++++++++------------ test/access-settings.test.js | 2 +- test/channel-settings-modal.test.js | 43 +++++++++++++++-------------- 6 files changed, 62 insertions(+), 49 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8af5826..16e4cb9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,8 @@ product overview. ## Unreleased +- Slack **⚙️ Settings** pages are tabs again: one row — General · Resume · MCP · Skills · Secrets — + with the open page highlighted, instead of the *Page* dropdown. - VS Code over SSH now starts in the channel's folder: File → Open Folder in a Remote-SSH window opens at the channel folder (or its custom folder) instead of the container's home. - Handing a generated file to Drive, Gmail or another Composio tool now goes through the gateway's diff --git a/FEATURES.md b/FEATURES.md index 929be08..736a3ce 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -364,9 +364,10 @@ A categorized catalog of what's shipped. Cross-linked to `TEST-PLAN.md` checks. - **Slack settings for authorized users:** replies requested by anyone allowed to use the agent add a requester-bound **⚙️ Settings** footer button. Its Block Kit console mirrors the web setup concepts across five pages — **General Settings**, **Resume Session**, **MCP**, **Skills**, - **Secrets** — picked from a single *Page* dropdown rather than a row of buttons that wrapped onto - a second line as pages were added. Legacy page ids (`runtime`, `access`, `network`) still resolve, - so a Settings view opened before the merge keeps navigating. + **Secrets** — shown as one row of tabs (*General · Resume · MCP · Skills · Secrets*) at the top, + the open page highlighted; short names keep the five on one row of the modal. Legacy page ids + (`runtime`, `access`, `network`) and the former *Page* dropdown still resolve, so a Settings view + opened before either change keeps navigating. **General Settings** is everything that decides how the conversation runs: its Engine & model scopes, then **Access** (with the network switch and the VPN row). Engine & model edits in place — six dropdowns, no nested form, each saving the diff --git a/TEST-PLAN.md b/TEST-PLAN.md index 5733964..01a6c8f 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -1971,8 +1971,8 @@ Automated: `test/channel-memory.test.js`, `test/memory-search.test.js`, (`test/slack-progress.test.js`, `test/channel-settings-modal.test.js`, `test/deliver.test.js`). - [ ] Live General Settings (engine-independent Slack UI case): in a disposable channel with a manager actor and an ordinary approved member, open **⚙️ Settings** from a reply. Pass when - the modal opens on General Settings with one *Page* dropdown listing five pages; switching - pages through it repaints in place; the manager sees Engine & model, Access (summary + + the modal opens on General Settings with one row of five tabs (General highlighted) that + fits on one line; clicking a tab repaints in place and highlights it; the manager sees Engine & model, Access (summary + *Change access settings*) and the VPN row under the network switch on one page, while the member sees the same page without the access summary and cannot reach the editor; saving access settings returns to General Settings with its notice; and a channel with a provisioned VPN shows *Checking @@ -1993,10 +1993,12 @@ Automated: `test/channel-memory.test.js`, `test/memory-search.test.js`, pasting the same line back as `/resume ` continues it from the thread. Then `/clear` the thread and reopen the tab: it must say there is no session rather than offering the cleared id. -- [x] Automated page dropdown and General Settings: the modal carries exactly one *Page* control — - a `static_select` whose options are General Settings / Resume Session / MCP / Skills / - Secrets in that order, opening on the page being shown, each option bound to the view's - channel and owner, and no page rendered as a button any more. General Settings carries the +- [x] Automated page tabs and General Settings: the modal carries one `actions` row of tab buttons + General / Resume / MCP / Skills / Secrets in that order, above the page content, exactly the + open page styled primary, each bound to the view's channel and owner, and no page dropdown; + a picked option from a view opened while the dropdown shipped still switches pages + (`test/channel-settings-modal.test.js`). Earlier the pages were a *Page* dropdown (Tiberiu + asked for tabs back, QA-0925). General Settings carries the Engine & model and Access headers in that order, with the VPN rendered as a single row immediately after the Auto/Lean/Network checkboxes rather than a section of its own. Every legacy page id (`runtime`, `access`, `network`) and an unknown one resolve to `general`, so a Settings view diff --git a/src/slack/channel-settings.js b/src/slack/channel-settings.js index c42933f..782f3d1 100644 --- a/src/slack/channel-settings.js +++ b/src/slack/channel-settings.js @@ -49,7 +49,7 @@ export const CHANNEL_SETTINGS_SECRETS_MANAGE_ACTION_ID = "cg_channel_settings_se export const CHANNEL_SETTINGS_VPN_TOGGLE_ACTION_ID = "cg_channel_settings_vpn_toggle"; export const CHANNEL_SETTINGS_VPN_REFRESH_ACTION_ID = "cg_channel_settings_vpn_refresh"; export const CHANNEL_SETTINGS_ACTION_PATTERN = /^cg_channel_settings(?:$|_)/; -// The pages the modal offers, in the order the dropdown lists them. "general" absorbed the former +// The pages the modal offers, in the order the tab row shows them. "general" absorbed the former // runtime, access and network tabs (see generalBlocks); LEGACY_TABS keeps a Settings view opened // before that merge — its buttons still carry the old ids — landing on the page that now owns // those controls instead of silently falling back to the first one. @@ -597,24 +597,31 @@ const TAB_LABELS = Object.freeze({ secrets: "Secrets", }); -// Pages are chosen from a dropdown rather than a row of buttons: an actions row wraps onto a -// second line in a narrow modal, and every page added made it worse. The select carries the same -// `tab` command the buttons did, so a Settings view opened before this shipped keeps switching -// pages through the very same handler. -function tabSelect(state, active) { - const options = CHANNEL_SETTINGS_TABS.map((tab) => - option(TAB_LABELS[tab], actionValue("tab", { c: state.channelId, u: state.ownerId, p: tab }))); +// The short names the tab row shows; TAB_LABELS stays the page's full name. +const TAB_BUTTON_LABELS = Object.freeze({ + general: "General", + resume: "Resume", + mcp: "MCP", + skills: "Skills", + secrets: "Secrets", +}); + +// Pages are a row of tab buttons, the current one highlighted. A dropdown replaced an earlier row +// that wrapped onto a second line as pages were added; with five short names one row fits a +// modal, and a tab is one click where the dropdown was two. Each button carries the same `tab` +// command the dropdown did, so a Settings view opened before this shipped still switches pages +// through the very same handler. +function tabRow(state, active) { return { - type: "section", + type: "actions", block_id: "cg_channel_settings_tabs", - text: mrkdwn("*Page*"), - accessory: { - type: "static_select", - action_id: CHANNEL_SETTINGS_TAB_SELECT_ACTION_ID, - placeholder: plain("Choose a page"), - options, - initial_option: options[Math.max(0, CHANNEL_SETTINGS_TABS.indexOf(active))], - }, + elements: CHANNEL_SETTINGS_TABS.map((tab) => ({ + type: "button", + action_id: `${CHANNEL_SETTINGS_TAB_PREFIX}${tab}`, + text: plain(TAB_BUTTON_LABELS[tab] || TAB_LABELS[tab]), + value: actionValue("tab", { c: state.channelId, u: state.ownerId, p: tab }), + ...(tab === active ? { style: "primary" } : {}), + })), }; } @@ -651,7 +658,7 @@ export function buildChannelSettingsView(snapshot = {}, state = {}, { blocks: [ { type: "context", elements: [mrkdwn(`Settings for *#${escapeMrkdwn(channelName || "this channel")}*. Anyone authorized to use the agent here can edit these settings. Access settings and VPN controls require a channel manager or admin. Cloud MCP is admin-only.`)] }, ...(notice ? [{ type: "section", text: mrkdwn(notice) }] : []), - tabSelect(state, active), + tabRow(state, active), { type: "divider" }, ...content, ], diff --git a/test/access-settings.test.js b/test/access-settings.test.js index 0f774d3..7791f7f 100644 --- a/test/access-settings.test.js +++ b/test/access-settings.test.js @@ -47,7 +47,7 @@ test("only admins and current managers see the Access section of General Setting const headers = view.blocks.filter((block) => block.type === "header").map((block) => block.text.text); assert.equal(headers.includes("Access"), !meta.isDM); assert.equal(JSON.stringify(view).includes("Who may use and manage this channel is shown"), !expected && !meta.isDM); - const pages = view.blocks.find((block) => block.block_id === "cg_channel_settings_tabs").accessory.options; + const pages = view.blocks.find((block) => block.block_id === "cg_channel_settings_tabs").elements; assert.ok(pages.some((page) => JSON.parse(page.value).p === "secrets")); } }); diff --git a/test/channel-settings-modal.test.js b/test/channel-settings-modal.test.js index 4c3fbc8..b0a0285 100644 --- a/test/channel-settings-modal.test.js +++ b/test/channel-settings-modal.test.js @@ -134,11 +134,9 @@ const snapshot = { }; const allButtons = (view) => view.blocks.flatMap((block) => block.elements || []).filter((item) => item.type === "button"); -const pagePicker = (view) => view.blocks.find((block) => block.block_id === "cg_channel_settings_tabs").accessory; -// The page dropdown is navigation, not a setting; every count below is about the controls that -// save something. -const selects = (view) => view.blocks.filter((block) => block.accessory?.type === "static_select" - && block.block_id !== "cg_channel_settings_tabs"); +const tabRow = (view) => view.blocks.find((block) => block.block_id === "cg_channel_settings_tabs"); +// Every count below is about the controls that save something; the tab row is navigation. +const selects = (view) => view.blocks.filter((block) => block.accessory?.type === "static_select"); const rendered = (view) => JSON.stringify(view); test("Settings footer button is authorized-user-only and requester-bound", () => { @@ -167,21 +165,21 @@ test("authorized user reply footer adds Settings after the existing workspace co assert.equal(ordinary.some((button) => button.action_id === CHANNEL_SETTINGS_ACTION_ID), false); }); -test("Channel Settings pages are chosen from one dropdown that shows the open page", () => { +test("Channel Settings pages are one row of tabs with the open page highlighted", () => { const view = buildChannelSettingsView(snapshot, state, { channelName: "project-alpha", tab: "mcp", canManageCloudMcp: true, canEditAccess: true }); - const picker = pagePicker(view); - assert.equal(picker.action_id, CHANNEL_SETTINGS_TAB_SELECT_ACTION_ID); - assert.deepEqual(picker.options.map((entry) => parseActionValue(entry.value).p), [...CHANNEL_SETTINGS_TABS]); - assert.deepEqual(picker.options.map((entry) => entry.text.text), - ["General Settings", "Resume Session", "MCP", "Skills", "Secrets"]); - // The dropdown opens on the page being shown, and every option is bound to this view's owner. - assert.equal(parseActionValue(picker.initial_option.value).p, "mcp"); - for (const entry of picker.options) { - assert.deepEqual({ ...parseActionValue(entry.value), p: undefined }, - { o: "tab", c: state.channelId, u: state.ownerId, p: undefined }); + const row = tabRow(view); + assert.equal(row.type, "actions", "tabs, not a dropdown (Tiberiu, QA-0925)"); + assert.equal(view.blocks.indexOf(row) < view.blocks.findIndex((block) => block.type === "header" || block.type === "section" && block !== row && /MCP connections/.test(JSON.stringify(block))), true, "above the page content"); + assert.deepEqual(row.elements.map((button) => parseActionValue(button.value).p), [...CHANNEL_SETTINGS_TABS]); + assert.deepEqual(row.elements.map((button) => button.text.text), ["General", "Resume", "MCP", "Skills", "Secrets"], + "short names, so the five fit one row of a modal"); + assert.deepEqual(row.elements.map((button) => button.action_id), CHANNEL_SETTINGS_TABS.map((tab) => `cg_channel_settings_tab_${tab}`)); + // Exactly the open page is highlighted, and every tab is bound to this view's owner. + assert.deepEqual(row.elements.filter((button) => button.style === "primary").map((button) => parseActionValue(button.value).p), ["mcp"]); + for (const button of row.elements) { + assert.deepEqual({ ...parseActionValue(button.value), p: undefined }, { o: "tab", c: state.channelId, u: state.ownerId, p: undefined }); } - // No page is a button any more, so the row cannot wrap onto a second line. - assert.equal(allButtons(view).some((button) => button.action_id.startsWith("cg_channel_settings_tab_")), false); + assert.equal(view.blocks.some((block) => block.accessory?.action_id === CHANNEL_SETTINGS_TAB_SELECT_ACTION_ID), false, "no page dropdown"); assert.match(rendered(view), /MCP connections/); assert.match(rendered(view), /Cloud MCP/); assert.match(rendered(view), /github/); @@ -191,8 +189,11 @@ test("Channel Settings pages are chosen from one dropdown that shows the open pa }); test("a settings control's command is read from a button value or from the picked option", () => { - const picked = pagePicker(buildChannelSettingsView(snapshot, state, { tab: "general" })).options[2]; - assert.deepEqual(settingsCommand({ selected_option: picked }), { o: "tab", c: state.channelId, u: state.ownerId, p: "mcp" }); + const tab = tabRow(buildChannelSettingsView(snapshot, state, { tab: "general" })).elements[2]; + assert.deepEqual(settingsCommand({ value: tab.value }), { o: "tab", c: state.channelId, u: state.ownerId, p: "mcp" }); + // A Settings view opened before tabs came back still carries the dropdown: its picked option + // switches pages through the same command. + assert.deepEqual(settingsCommand({ selected_option: { value: tab.value } }), { o: "tab", c: state.channelId, u: state.ownerId, p: "mcp" }); assert.deepEqual(settingsCommand({ value: actionValue("tab", { p: "skills" }) }), { o: "tab", p: "skills" }); // A runtime dropdown's bare model id is not a command, and neither is a missing control. assert.deepEqual(settingsCommand({ selected_option: { value: "claude-opus-4-8" } }), {}); @@ -264,7 +265,7 @@ test("Resume Session tab shows this thread's copyable command, and says why when assert.match(text, /\/resume /); assert.match(text, /994f6108-b405-4bbe-b96d-1641051df2fa/); assert.equal(parseSettingsMetadata(view.private_metadata).tab, "resume"); - assert.ok(pagePicker(view).options.some((entry) => parseActionValue(entry.value).p === "resume"), "the page dropdown offers it"); + assert.ok(tabRow(view).elements.some((button) => parseActionValue(button.value).p === "resume" && button.style === "primary"), "its tab is the highlighted one"); const noSession = rendered(buildChannelSettingsView({ ...snapshot, resume: { inThread: true, command: "" } }, state, { tab: "resume" })); assert.match(noSession, /No session in this thread yet/); From a5182751735befb2ce114c7ca5b1ce2b15bb8b67 Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 25 Sep 2026 14:57:35 +0300 Subject: [PATCH 12/14] feat: sync the whole channel folder with Drive, in a confined container Google Drive sync used to mirror only a Drive/ subfolder of the channel. The owner wants the channel itself in Drive (QA-0925), so the whole work folder is now the local side, with a filters file (case-insensitive, both directions) that keeps out agent instructions and skills at any depth, channel memory, secrets and key files, .git, dependency trees and rclone link stand-ins; a .driveignore file adds a channel's own exclusions and can only ever narrow the sync. An independent review showed that rclone on the host would write Drive files THROUGH a symlink in the work folder - to anywhere the daemon user can write - which the old subfolder sync allowed too. rclone now runs in a one-shot container (podman run --rm, caps dropped, --init, named for forced removal on timeout) that mounts only the work folder at its real path and the sync state, filters and key at a random path per pass; every symlink found is also excluded for the pass. A folder that is or contains the home, the gateway root or the workspace root, a hidden folder of the home, or a workspace folder other than the channel's own is refused. The completed resync now records what it was made for (local root, Drive folder, filters); changing any of them forces a fresh resync from a clean state dir instead of a bisync against the old pair's listings. Co-Authored-By: Claude Opus 5.5 (1M context) Signed-off-by: Tiberiu Socaci --- CHANGELOG.md | 6 + FEATURES.md | 23 +- TEST-PLAN.md | 24 +- public/index.html | 4 +- src/gateway/drivesync.js | 236 ++++++++++++++++-- .../references/administration.md | 5 +- src/mcp/tools/channel-admin.js | 15 +- src/runtimes/container/index.js | 29 +++ test/drivesync-manual.test.js | 5 +- test/drivesync.test.js | 162 +++++++++++- 10 files changed, 465 insertions(+), 44 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8af5826..4733e7a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,12 @@ product overview. ## Unreleased +- Google Drive sync now syncs the channel's whole folder instead of a `Drive/` subfolder. Agent + instructions and skills, channel memory, secrets, `.git` and dependency trees never sync, a + `.driveignore` file adds a channel's own exclusions, and symlinks are never followed. rclone now + runs in a throwaway container that sees only the channel folder, so a planted link can never + write onto the host (this also closes that hole in the old subfolder sync). A folder that + contains the home or the gateway's data is refused. - VS Code over SSH now starts in the channel's folder: File → Open Folder in a Remote-SSH window opens at the channel folder (or its custom folder) instead of the container's home. - Handing a generated file to Drive, Gmail or another Composio tool now goes through the gateway's diff --git a/FEATURES.md b/FEATURES.md index 929be08..6a81b91 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -2990,9 +2990,26 @@ are retired, bullet by bullet; everything else stands. pending (an armed *clear* toggle, a typed password or key), because a pending action captured as "already saved" would silently never run. → TEST-PLAN: Admin UI. - **Google Drive two-way sync (scheduled)**: a per-channel Drive folder link (channel settings) - is bisync'd on a timer into a dedicated `Drive/` subfolder of that channel's working folder — - never the folder root, so the confinement scaffolding (`.claude/`, `CLAUDE.md`, `MEMORY.md`, - `memory/`, `uploads/`) is never synced or overwritten. Auth is a Workspace service account + is bisync'd on a timer with that channel's WHOLE working folder (it used to be a `Drive/` + subfolder; QA-0925). A filters file, matched case-insensitively and applied in both directions, + keeps out agent instructions, skills and MCP config at any depth (`CLAUDE.md`, `AGENTS.md`, + `AGENTS.override.md`, `CLAUDE.local.md`, `.mcp.json`, `.claude/`, `.agents/`, `.codex/`), channel + memory (`MEMORY.md`, `memory/`), secrets (`.env*`, `.ssh/`, key and credential files, per-run env + folders), `.git` (file or folder), `.worktrees`, dependency trees and `*.rclonelink`, so a Drive + editor cannot plant instructions for the agent and no local secret is pushed. A `.driveignore` + at the folder root adds the channel's own exclusions (every line becomes an exclude). Every + symlink in the folder is excluded for the pass, and rclone runs in a throwaway container + (`podman run --rm`, all capabilities dropped) that mounts only the channel folder at its real + path and the sync state, filters (read-only) and key (read-only) at a random path per pass: a + link planted to anywhere else — even mid-pass — resolves inside that container and can never + write onto the host or into the sync state. A timed-out pass force-removes its container. A folder + that is or contains the operator's home, the gateway root or the workspace root, a hidden + configuration folder of the home (`~/.ssh`, `~/.config`, …), or a workspace folder that is not + this channel's own (another channel's, the platform parent, `.runtime`) is refused — so is a Lean + (clean-mode) channel, whose runs use a bare folder inside the gateway root. The first sync of a link merges both sides, + the newer copy winning where both hold the same path; the completed `--resync` records what it + was made for (local root, Drive folder, filters), and changing any of them makes the next pass a + fresh `--resync` from a clean state dir instead of a bisync against the old pair's listings. Auth is a Workspace service account (+ optional domain-wide-delegation subject), passed to `rclone bisync` via flags — no interactive `rclone config`, no per-user OAuth, no secret in the child env. The key is entered by **pasting the service-account JSON** into the settings page: stored write-only (validated as a real SA key, diff --git a/TEST-PLAN.md b/TEST-PLAN.md index 5733964..f507aa9 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -3280,10 +3280,26 @@ structural invariants are automated; rendered navigation and feature claims also Test action no-op with a clear message and never throw (smoke-tested). - [ ] Manual (needs rclone + a Workspace service-account key): set the global key-file path + enable; set a channel's Drive folder link; click **Test** → "Connected". Then wait one - interval (or restart) → files appear in `/Drive/`; a local edit there - propagates up to Drive and a Drive edit propagates down, on the next tick. -- [ ] Confinement: the sync only ever writes under `Drive/` — `.claude/`, `CLAUDE.md`, `AGENTS.md`, - `MEMORY.md`, `memory/`, `uploads/` are never pushed to Drive nor overwritten from it. + interval (or restart) → the channel folder's files appear in the Drive folder and Drive files + appear in the channel folder; a local edit propagates up and a Drive edit down, on the next tick. +- [x] Confinement (`test/drivesync.test.js`, real rclone): the whole folder syncs, while `.claude/`, + `CLAUDE.md`, `MEMORY.md`, `memory/`, `runtime/env/` and `.env` never reach Drive and a Drive-side + `CLAUDE.md` never overwrites the channel's; `.driveignore` lines only ever add excludes; filters + carry `--ignore-case`; every symlink is excluded for the pass (names glob-escaped); a folder that + is or contains the home, the gateway root or the workspace root, a hidden folder of the home, or a + workspace folder other than this channel's own is refused; the pass is launched as `podman run + --rm --pull=never --cap-drop ALL --name cg-drivesync-…` with the work folder at its real path and + the state, read-only filters and read-only key at a fresh random `/cg-sync-` path per pass + (no host path of the state or key exists inside), and a timed-out pass force-removes it; a changed local root, Drive folder or filter set forces + a fresh `--resync` (a pre-identity sentinel from the `Drive/` subfolder era resyncs once). + Live-verified on Xavier (QA-0925 review): with a work-folder symlink to a host folder and a + Drive-side payload under it, the confined pass left the host folder empty even without the + symlink exclude; with it, the pass succeeded, `lnk2 -> .claude` could not overwrite `.claude/`, + and Drive-side `Claude.md` / `AGENTS.override.md` did not come down. +- [ ] Live (engine-independent): on a channel whose folder has a lockdown, memory and a `.env`, link + a Drive folder and **Sync now**. Pass: the Drive folder holds the channel's own files but none of + `.claude/`, `CLAUDE.md`, `AGENTS.md`, `MEMORY.md`, `memory/`, `.env`, `.git`; a file added in Drive + reaches the channel folder; a pattern added to `.driveignore` stops that path syncing. - [ ] A failed first run leaves no half-baked bisync state (the state dir is dropped, so the next tick retries with `--resync`). - [x] A prior listing that records no file (`.lst`, or the `.lst-err` rclone set aside after refusing diff --git a/public/index.html b/public/index.html index 0f81f37..5c7e100 100644 --- a/public/index.html +++ b/public/index.html @@ -688,7 +688,7 @@

Public file links

Google Drive sync

-

Two-way sync (scheduled, via rclone bisync) between a per-channel Google Drive folder and a dedicated Drive/ subfolder inside that channel's working folder. Auth is a Workspace service account — drop its JSON key file on this host and point to it below. Set a channel's folder link under Conversations → the channel → Google Drive sync folder. rclone must be installed on this host (apt install rclone, or the official installer from rclone.org).

+

Two-way sync (scheduled, via rclone bisync) between a per-channel Google Drive folder and that channel's whole working folder — except agent instructions and skills (CLAUDE.md, AGENTS.md, .claude/, …), channel memory, secrets (.env, keys), .git and dependency trees, plus anything listed in the folder's .driveignore. Symlinks are never followed, and rclone runs in a throwaway container that sees only the channel folder. Auth is a Workspace service account — drop its JSON key file on this host and point to it below. Set a channel's folder link under Conversations → the channel → Google Drive sync folder. rclone must be installed on this host (apt install rclone, or the official installer from rclone.org).