From 4686e84b630084e0a785a40964c29556b51e5914 Mon Sep 17 00:00:00 2001 From: Tiberiu Socaci Date: Fri, 23 Oct 2026 18:35:20 +0300 Subject: [PATCH] feat: add per-harness default effort settings Signed-off-by: Tiberiu Socaci --- CHANGELOG.md | 4 ++ FEATURES.md | 6 +++ TEST-PLAN.md | 23 +++++++++ docs/ENGINE-CAPABILITIES.md | 2 +- .../docs/configuration/model-defaults.md | 6 ++- public/app.js | 22 ++++++-- public/index.html | 15 ++++++ src/config/settings.js | 11 ++++ src/engines/adapters.js | 8 +-- src/engines/contract.js | 2 +- src/engines/qwen.js | 6 +-- src/gateway/run.js | 15 ++++-- src/web/routes/settings.js | 19 ++++++- test/default-model.test.js | 19 ++++++- test/engine-adapter-contract.test.js | 2 +- test/engine-registry.test.js | 2 +- test/qwen-provider-settings-browser.test.js | 21 +++++++- test/qwen-provider-settings-route.test.js | 49 ++++++++++++++++++ test/runtime-identity-preamble.test.js | 50 ++++++++++++++++++- 19 files changed, 255 insertions(+), 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c34b0f62..9279a65d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,10 @@ product overview. - Supply new installations with Astra, Sol, Terra, Fable, Opus and Sonnet (`:sonet`) shortcuts using the newest supported stable family versions. Preserve customized lists and thread pins. +- Add a default effort selector beside each harness's default model, including Qwen providers. + Inherited runs use it unless a conversation, thread or request overrides effort; blank keeps + the CLI default. Validate model compatibility and use the destination harness's effort on failover. + - Let organization admins use another requester's Slack menu buttons and question forms. Menu views open under the clicking admin's own identity; question answers continue the original requester's session and record which admin responded. Current access, membership and stale-form diff --git a/FEATURES.md b/FEATURES.md index d0c9e5be..ea785544 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -2458,6 +2458,12 @@ A categorized catalog of what's shipped. Cross-linked to `TEST-PLAN.md` checks. logins stay on Codex. Unknown, disabled or invalid targets and missing request text are refused before a child thread is posted. A running or sudo/source-host session is refused until it can be forked safely. +- Gateway-wide **default effort** beside each harness's default model (Settings → Agent defaults → + Engine & runtime, including each Qwen provider card). Choices follow the selected model's + supported effort levels; saves validate the model/effort pair atomically. Per-request, thread + and channel/DM-template effort overrides take precedence. Inherited fresh and resumed turns use + the harness default; blank preserves the CLI default. Failover uses the destination harness's + own default effort, and model retries recheck compatibility before passing an effort flag. - Gateway-wide **default model** per engine (Settings → Engine & runtime, validated like `/model`): when a thread/channel/DM-template sets no model, the run gets the gateway default as an explicit `--model`/`-m` — so the admin's interactive terminal `/model` choice (written to diff --git a/TEST-PLAN.md b/TEST-PLAN.md index b685ddb6..109a0f80 100644 --- a/TEST-PLAN.md +++ b/TEST-PLAN.md @@ -1,5 +1,28 @@ # ChannelGate — Test Plan +## Harness default effort (2026-10-23) + +- [x] Automated: `node --test test/default-model.test.js test/qwen-provider-settings-route.test.js + test/engine-registry.test.js test/engine-adapter-contract.test.js + test/runtime-identity-preamble.test.js` — settings round-trip, legacy/invalid/blank handling, + atomic API validation for every harness, fresh/resumed CLI flags, request > thread > channel > + gateway > CLI precedence, plus destination defaults through failover and model retry. +- [x] Browser: `CG_BROWSER_MODULE=/absolute/path/to/playwright/index.mjs node --test + test/qwen-provider-settings-browser.test.js` — disposable admin API and real Chromium; + save/reload/clear distinct Claude, Codex and Qwen EU efforts without changing production settings. +- [ ] Live Claude and Codex acceptance: for each engine, use a private read fixture with that + engine selected and model/effort unset on the channel and thread. Record existing gateway + defaults; in Admin UI → Agent defaults set the engine's model and effort to a supported pair + (Claude sonnet/high; Codex gpt-6-sol/medium). Start a fresh thread as an approved test actor: + “Which configured engine, model and effort are you running? One line.” Check captured runtime + facts and actual engine flags, not just the answer. Change gateway effort to low, send “And + now?” in the same thread, and verify the resumed turn receives low. Set channel effort medium, + then thread effort high, then request effort low through the Run API; each more specific choice + must win. Clear each override and the gateway effort; verify the effort flag is omitted. Restore + the recorded defaults. PASS only when actual spawn evidence agrees at every step for both + engines. Repeat replay-safe failover with different source/destination defaults and verify the + destination uses its own effort. Pending live execution; automated stubs are not live engine PASS. + ## Admin access to Slack buttons and forms (2026-10-23) - [x] Automated: `node --test test/admin-menu-controls.test.js test/file-button-actions.test.js diff --git a/docs/ENGINE-CAPABILITIES.md b/docs/ENGINE-CAPABILITIES.md index efaee2d4..847a6c73 100644 --- a/docs/ENGINE-CAPABILITIES.md +++ b/docs/ENGINE-CAPABILITIES.md @@ -21,7 +21,7 @@ The Qwen column covers every harness generated from the Anthropic-compatible **p `src/engines/qwen.js` — today `qwen` (QwenCloud) and `qwen-eu` (Alibaba Cloud Model Studio, EU region). They share this column because they share the CLI and every guarantee in it; what differs per entry is the endpoint, the credential, the shipped and discovered model catalog, and the -gateway default model. +gateway default model and effort. Each is **opt-in**: a missing `engineEnabled` entry means OFF, it is terminal in the failover graph in both directions, and its provider credential (its own `…ApiKey` / `…BaseUrl` pair) is gateway diff --git a/documentation/src/content/docs/configuration/model-defaults.md b/documentation/src/content/docs/configuration/model-defaults.md index 40d5eca4..664692a0 100644 --- a/documentation/src/content/docs/configuration/model-defaults.md +++ b/documentation/src/content/docs/configuration/model-defaults.md @@ -15,6 +15,8 @@ Open **Settings → Agent defaults → Engine & runtime**. | Engine (which CLI drives the bot) | Default engine; Claude is the fallback default when none is configured | | Default Claude model | Sets the model used by inherited Claude runs | | Default Codex model | Sets the model used by inherited Codex runs | +| Default Claude / Codex effort | Sets reasoning effort when a conversation or thread does not override it | +| Provider default model and effort | Independent defaults on each optional Qwen provider card | | Automatic failover between harnesses | Enables supported Claude–Codex recovery before work starts; defaults on | | How a failover happens | Automatic, or ask in a live Slack thread; automatic is the default | @@ -22,7 +24,7 @@ Optional provider harnesses must be explicitly enabled and authenticated. At lea 1. Authenticate the engines you intend to use. 2. Disable engines the deployment cannot support. -3. Choose the default engine and an explicit model for each enabled engine when its controls offer one. +3. Choose the default engine, model, and supported effort for each enabled engine when its controls offer them. 4. Save, then create a fresh thread to check inherited behavior. An empty gateway model setting lets the CLI use its own default. An explicit gateway model helps avoid changes in the operator's interactive terminal model choice affecting gateway runs. @@ -47,7 +49,7 @@ Set a conversation's **Engine**, **Model**, and **Effort** under **Conversations For a new run, explicit per-run overrides take precedence, followed by thread selection, conversation/DM selection, and gateway defaults. Existing threads retain their owning engine session unless explicitly switched or recovered; changing the global default does not automatically migrate every active thread. -Effort is engine- and model-dependent. Use the offered choices rather than assuming all models support `xhigh` or the same reasoning settings. An empty effort value inherits the engine's default; it is not proof of a specific reasoning level. +Effort is engine- and model-dependent. Use the offered choices rather than assuming all models support `xhigh` or the same reasoning settings. An empty conversation or thread effort inherits the gateway's default effort for that harness. An empty gateway effort uses the CLI default; it is not proof of a specific reasoning level. Failover uses the destination harness's own gateway effort, checked against its selected model. ## Control who changes runtime diff --git a/public/app.js b/public/app.js index 0aca8faf..e4c762c0 100644 --- a/public/app.js +++ b/public/app.js @@ -128,14 +128,14 @@ function engineOptionsHtml({ includeDefault = false } = {}) { selectableEngines().map((m) => ``).join(""); } -function syncEffortOptions({ engineSelect, modelSelect, effortSelect, label, value }) { - const engine = effectiveEngine(engineSelect?.value || ""); +function syncEffortOptions({ engineSelect, modelSelect, effortSelect, label, value, engine: pinnedEngine, blankLabel = "default" }) { + const engine = pinnedEngine || effectiveEngine(engineSelect?.value || ""); const modelEfforts = MODEL_EFFORT_OPTIONS[engine]?.[modelSelect?.value || ""]; const options = modelEfforts?.length ? [["", "default"], ...modelEfforts.map((effort) => [effort, effort])] : EFFORT_OPTIONS[engine] || EFFORT_OPTIONS.claude; const current = value !== undefined ? value : effortSelect.value; - effortSelect.innerHTML = options.map(([v, text]) => ``).join(""); + effortSelect.innerHTML = options.map(([v, text]) => ``).join(""); effortSelect.value = options.some(([v]) => v === current) ? current : ""; if (label) label.textContent = `Effort (${ENGINE_MANIFESTS.find((m) => m.id === engine)?.label || engine})`; } @@ -187,6 +187,12 @@ function syncModelOptions({ engineSelect, modelSelect, value, engine, blankLabel modelSelect.value = options.some(([v]) => v === current) ? current : ""; } +function syncDefaultEffort(modelSelect, effortSelect, engine, value) { + const sync = (next) => syncEffortOptions({ modelSelect, effortSelect, engine, value: next, blankLabel: "CLI default" }); + sync(value || ""); + modelSelect.onchange = () => sync(effortSelect.value); +} + function addModelShortcutRow(name = "", target = {}) { const box = document.getElementById("model-shortcuts-editor"); const row = document.createElement("div"); @@ -3769,6 +3775,7 @@ function paintQwenProviders(providers) { const keyInput = at("api-key"); const urlInput = at("base-url"); const modelSelect = at("default-model"); + const effortSelect = at("default-effort"); const clearBtn = at("clear-key"); // An endpoint the provider ships is a placeholder (blank = keep it); one only the operator can // know is not, so the box says so instead of implying a default exists. @@ -3779,6 +3786,7 @@ function paintQwenProviders(providers) { // The button sits inside the field's + +
diff --git a/src/config/settings.js b/src/config/settings.js index c779de02..44808a18 100644 --- a/src/config/settings.js +++ b/src/config/settings.js @@ -334,6 +334,14 @@ export function getDefaultModel(engine) { return typeof v === "string" ? v.trim() : ""; } +// Blank preserves the CLI default; stale or hand-edited values never reach a runner. +export function getDefaultEffort(engine) { + const adapter = adapterOr(engine); + const raw = getSettings()[adapter.defaultEffortKey]; + const effort = typeof raw === "string" ? raw.trim().toLowerCase() : ""; + return adapter.efforts.includes(effort) ? effort : ""; +} + export function getModelShortcuts() { return modelShortcutsFromSettings(getSettings(), { claude: modelsForEngine("claude"), codex: modelsForEngine("codex") }); } @@ -493,12 +501,14 @@ export function qwenProviderSettings() { apiKeyField: entry.settings.apiKey, baseUrlField: entry.settings.baseUrl, defaultModelField: entry.defaultModelKey, + defaultEffortField: entry.defaultEffortKey, hasApiKey: Boolean(config.apiKey), apiKeyLast4: last4(config.apiKey), // The STORED endpoint, so an empty box means "still on the shipped default" rather than // silently re-saving a default the operator never chose. baseUrl: typeof getSettings()[entry.settings.baseUrl] === "string" ? getSettings()[entry.settings.baseUrl].trim() : "", defaultModel: getDefaultModel(entry.id), + defaultEffort: getDefaultEffort(entry.id), }; }); } @@ -1153,6 +1163,7 @@ export function settingsForApi() { engine: getEngine(), defaultClaudeModel: getDefaultModel("claude"), defaultCodexModel: getDefaultModel("codex"), + ...Object.fromEntries(ENGINES.map((id) => [adapterOr(id).defaultEffortKey, getDefaultEffort(id)])), // One entry per Anthropic-compatible provider, so the admin UI renders a card per harness // instead of carrying a hard-coded copy of the table. qwenProviders: qwenProviderSettings(), diff --git a/src/engines/adapters.js b/src/engines/adapters.js index bcbe4ed0..d715ec90 100644 --- a/src/engines/adapters.js +++ b/src/engines/adapters.js @@ -59,7 +59,7 @@ const baseCompile = (engine, request = {}, supportedModes = ["off"]) => { const claude = validateEngineAdapter({ pluginCapabilities: { manifest: "claude", components: ["skills", "commands", "agents", "hooks", "mcpServers"] }, resolveOptionalMcpConfig: resolveClaudeMcpConfig, - id: "claude", label: "Claude", cli: "claude", defaultModelKey: "defaultClaudeModel", mcpMetaKey: "allowedMcps", + id: "claude", label: "Claude", cli: "claude", defaultModelKey: "defaultClaudeModel", defaultEffortKey: "defaultClaudeEffort", mcpMetaKey: "allowedMcps", instructionFile: "CLAUDE.md", skillsDir: ".claude/skills", mcpTransport: "file", contextWindow: 200_000, efforts: ["low", "medium", "high", "xhigh"], models: [ { label: "Opus 5.5", value: "claude-opus-5-5", description: "Current Opus model for most coding and knowledge work." }, @@ -205,7 +205,7 @@ const claude = validateEngineAdapter({ const makeQwenAdapter = (entry) => validateEngineAdapter({ pluginCapabilities: { manifest: "claude", components: ["skills", "commands", "agents", "hooks", "mcpServers"] }, resolveOptionalMcpConfig: resolveClaudeMcpConfig, - id: entry.id, label: entry.label, cli: "claude", defaultModelKey: entry.defaultModelKey, + id: entry.id, label: entry.label, cli: "claude", defaultModelKey: entry.defaultModelKey, defaultEffortKey: entry.defaultEffortKey, // The shared model-ID vocabulary these harnesses draw from (Qwen/GLM/DeepSeek/Kimi/MiniMax). // Declared so the admin UI can recognise a provider harness's model without a hard-coded engine // id — a provider added to the table would otherwise have its saved model silently dropped from @@ -312,7 +312,7 @@ const codex = validateEngineAdapter({ } return { servers, rejected }; }, - id: "codex", label: "Codex", cli: "codex", defaultModelKey: "defaultCodexModel", mcpMetaKey: "allowedCodexMcps", + id: "codex", label: "Codex", cli: "codex", defaultModelKey: "defaultCodexModel", defaultEffortKey: "defaultCodexEffort", mcpMetaKey: "allowedCodexMcps", // Codex reports no dollar cost, so the ledger prices it from this configured rate (Settings → // Audit). Declared here so no other engine inherits OpenAI's rates by falling through. costRateKey: "codexRatePer1MTokens", @@ -375,7 +375,7 @@ const codex = validateEngineAdapter({ }); const opencode = validateEngineAdapter({ - id: "opencode", label: "OpenCode (read-only)", cli: "opencode", defaultModelKey: "defaultOpenCodeModel", + id: "opencode", label: "OpenCode (read-only)", cli: "opencode", defaultModelKey: "defaultOpenCodeModel", defaultEffortKey: "defaultOpenCodeEffort", mcpMetaKey: "allowedOpenCodeMcps", instructionFile: "AGENTS.md", skillsDir: ".opencode/skills", mcpTransport: "none", contextWindow: 200_000, efforts: ["low", "medium", "high"], models: [], mintsOwnSessionId: true, diff --git a/src/engines/contract.js b/src/engines/contract.js index c873e7b1..f39eaad1 100644 --- a/src/engines/contract.js +++ b/src/engines/contract.js @@ -49,7 +49,7 @@ export function validateRunContext(context) { } const REQUIRED_METHODS = ["compileConfinement", "run", "interrupt", "discoverMcps", "health"]; -const REQUIRED_MANIFEST = ["id", "label", "cli", "defaultModelKey", "mcpMetaKey", "instructionFile", "skillsDir", "mcpTransport", "contextWindow"]; +const REQUIRED_MANIFEST = ["id", "label", "cli", "defaultModelKey", "defaultEffortKey", "mcpMetaKey", "instructionFile", "skillsDir", "mcpTransport", "contextWindow"]; export function validateEngineAdapter(adapter) { if (!adapter || typeof adapter !== "object") throw new TypeError("EngineAdapter must be an object"); diff --git a/src/engines/qwen.js b/src/engines/qwen.js index c9cac083..5c991559 100644 --- a/src/engines/qwen.js +++ b/src/engines/qwen.js @@ -54,7 +54,7 @@ export function qwenModelLabel(id) { * `label` what every picker shows * `harnessLabel` the short name a provider failure is attributed to (see claude.js) * `settings` the gateway settings keys holding this provider's credential and endpoint - * `defaultModelKey` the settings key holding this provider's gateway default model + * `defaultModelKey` / `defaultEffortKey` settings keys for this provider's gateway defaults * `defaultBaseUrl` shipped endpoint, or "" when only the operator can know it * `models` the catalog offered until the account's own list is read * `endpointHint` admin-UI copy: what a valid endpoint for this provider looks like @@ -73,7 +73,7 @@ export const QWEN_PROVIDERS = Object.freeze([ harnessLabel: "Qwen", catalogName: "QwenCloud", settings: Object.freeze({ apiKey: "qwenApiKey", baseUrl: "qwenBaseUrl" }), - defaultModelKey: "defaultQwenModel", + defaultModelKey: "defaultQwenModel", defaultEffortKey: "defaultQwenEffort", defaultBaseUrl: QWEN_DEFAULT_BASE_URL, description: "The same Claude Code CLI, pointed at QwenCloud's Anthropic-compatible endpoint.", endpointHint: "Token Plan keys use token-plan.maas.qwencloudapi.com; pay-as-you-go keys use maas.qwencloudapi.com.", @@ -85,7 +85,7 @@ export const QWEN_PROVIDERS = Object.freeze([ harnessLabel: "Qwen EU", catalogName: "Model Studio", settings: Object.freeze({ apiKey: "qwenEuApiKey", baseUrl: "qwenEuBaseUrl" }), - defaultModelKey: "defaultQwenEuModel", + defaultModelKey: "defaultQwenEuModel", defaultEffortKey: "defaultQwenEuEffort", // Deliberately EMPTY. An EU Model Studio endpoint is per-WORKSPACE // (`https://ws-.eu-central-1.maas.aliyuncs.com/apps/anthropic`), so there is no // value this repository could ship that is correct for anyone but the account it was copied diff --git a/src/gateway/run.js b/src/gateway/run.js index 4f54abb9..266015f4 100644 --- a/src/gateway/run.js +++ b/src/gateway/run.js @@ -24,7 +24,7 @@ import { abortPooled } from "../engines/session-pool.js"; import { DEFAULT_SILENCE_WINDOWS } from "../engines/watchdog.js"; import { mintsOwnSessionId, usesMcpConfigFile, engineSupports, requireAdapter, fallbackTargets, engineLabel, engineCredentialState, engineTransientKinds, modelsForEngine } from "../engines/registry.js"; import { validateRunContext } from "../engines/contract.js"; -import { getEngine, getDefaultModel, getDmTemplate, getEngineFallback, isEngineEnabled, getEnabledEngines, ENGINES, getOrgAccessGrants } from "../config/settings.js"; +import { getEngine, getDefaultModel, getDefaultEffort, getDmTemplate, getEngineFallback, isEngineEnabled, getEnabledEngines, ENGINES, getOrgAccessGrants } from "../config/settings.js"; import { claudeTokenFingerprint, resolveContainerClaudeToken } from "./claude-token-relay.js"; import { installNestedClaudeLogin } from "./nested-claude-login.js"; import { installPerplexityResearchLogin, perplexityResearchPreamble } from "./perplexity-research.js"; @@ -45,7 +45,7 @@ import { runtimeAccessPreamble } from "./runtime-access.js"; import { channelCredentialsPreamble } from "./channel-credentials.js"; import { resolveRunIntegrations } from "./run-integrations.js"; import { customMcpTokens } from "./custom-mcps.js"; -import { modelBelongsToEngine, effortBelongsToEngine } from "../engines/registry.js"; +import { modelBelongsToEngine, effortBelongsToEngine, effortBelongsToModel } from "../engines/registry.js"; import { writeFile, rm, mkdir } from "node:fs/promises"; import path from "node:path"; import { gatewayRoot, runTmpDir, workspaceRoot } from "../config/paths.js"; @@ -1323,7 +1323,9 @@ async function runMessageImpl({ channelId, authorId, workspaceId = "", text, thr // precedence order that actually belongs to this engine wins; none → "" (the CLI's own default). const gatewayDefaultModel = getDefaultModel(engine); let model = [threadModel, meta.model, gatewayDefaultModel].find((m) => m && modelBelongsToEngine(m, engine)) || ""; - const effort = [threadEffort, meta.effort].find((e) => e && effortBelongsToEngine(e, engine)) || ""; + const gatewayDefaultEffort = getDefaultEffort(engine); + const effortForModel = (selectedModel) => [threadEffort, meta.effort, gatewayDefaultEffort] + .find((e) => e && effortBelongsToModel(e, engine, selectedModel)) || ""; // Dashboard-only lifecycle signal: by this point the exact spawn engine/model are known. Keep it // best-effort so a display-layer callback can never prevent an engine run. try { onRuntimeResolved?.({ engine, model, ...runtimeSignal }); } catch { /* non-fatal */ } @@ -1501,6 +1503,7 @@ async function runMessageImpl({ channelId, authorId, workspaceId = "", text, thr if (sudoQuestion && !(await isAdmin(authorId))) throw new Error("Sudo task authorization was revoked before the engine could start."); }; const runOnce = async (sid, fresh, promptOverride = null, modelOverride = model) => { + const effort = effortForModel(modelOverride); assertRuntimeCanStart(); await assertScheduledSudoAuthority(); assertChannelEngineApproved((await getChannelMeta(entry.slug)) ?? channelMeta, engine); @@ -1745,7 +1748,9 @@ async function runMessageImpl({ channelId, authorId, workspaceId = "", text, thr // could still refuse the fallback — so it never promises an engine that then does not run. if (note) announceAnswerNote(note); if (fbMcpDropNote) announceAnswerNote(fbMcpDropNote); + const fallbackDefaultEffort = getDefaultEffort(fallbackEngine); const exec = async (fbSid, fbFresh, modelOverride = fallbackModel) => { + const effort = effortBelongsToModel(fallbackDefaultEffort, fallbackEngine, modelOverride) ? fallbackDefaultEffort : ""; assertRuntimeCanStart(); await assertScheduledSudoAuthority(); assertChannelEngineApproved((await getChannelMeta(entry.slug)) ?? channelMeta, fallbackEngine); @@ -1753,7 +1758,7 @@ async function runMessageImpl({ channelId, authorId, workspaceId = "", text, thr return fallbackAdapter.run(validateRunContext({ principal: { kind: untrustedPrincipal ? "daemon" : PRINCIPAL_KIND_BY_ORIGIN[origin] || "daemon", id: authorId }, origin, cwd, prompt: (fbFresh ? memoryPrefix : "") + composioIdentityPrefix + channelCredentialsPrefix - + runtimeIdentityPreamble({ engine: fallbackEngine, model: modelOverride, effort: "", fresh: fbFresh }) + + runtimeIdentityPreamble({ engine: fallbackEngine, model: modelOverride, effort, fresh: fbFresh }) + runtimeAccessPreamble(target, { clean, allowNetwork: Boolean(meta.allowNetwork), dangerouslySkip }) + perplexityResearchPreamble(target, { clean }) + fbPrompt, session: { id: fbSid, fresh: fbFresh }, policy: fallbackConfinement, // The SAME runtime target: failing over to the other harness changes which CLI runs, not @@ -1769,7 +1774,7 @@ async function runMessageImpl({ channelId, authorId, workspaceId = "", text, thr settingsFile: runSettingsFile, permissionPromptTool, writable: codexWritable, autoApprove: codexAutoApprove, clean, composioUserEndpoint, composioEndpoint, composioUserToken, composioToken, toolboxToken, makeToolboxUrl, makeToolboxKey, customMcps, - gatewayCapability: fallbackMcpRuntime.gatewayCapability, gatewayFsRoot, gatewayWorkspaceRoot, progressReport: progressReportEnabled, model: modelOverride, effort: "", + gatewayCapability: fallbackMcpRuntime.gatewayCapability, gatewayFsRoot, gatewayWorkspaceRoot, progressReport: progressReportEnabled, model: modelOverride, effort, attachments, signal, timeoutMs, maxSilenceMs, onDelta: scopedOnDelta, onEvent: scopedOnEvent, onSessionResolved: sessionResolved(fbKey, fallbackEngine), channelEnv, channelEnvFingerprint: channelEnvFp, browserNamespace, diff --git a/src/web/routes/settings.js b/src/web/routes/settings.js index 3dd6cbb4..eee8355d 100644 --- a/src/web/routes/settings.js +++ b/src/web/routes/settings.js @@ -15,9 +15,10 @@ import { allowedFsRoot, resolveWithinRoot, pathWithin, hashPassword, verifyPassw import { listAvailableSkills } from "../../gateway/folders.js"; import { workspaceAssignmentsAtPath } from "../../gateway/workspace-assignments.js"; import { listChannels, getChannelsIndex } from "../../config/store.js"; -import { requireAdapter, modelBelongsToEngine } from "../../engines/registry.js"; +import { requireAdapter, modelBelongsToEngine, effortBelongsToModel } from "../../engines/registry.js"; import { getAdminPassword, + getSettings, settingsForApi, saveSettings, isPerplexitySessionToken, @@ -415,6 +416,22 @@ export function createSettingsRouter({ if (v && !modelBelongsToEngine(v, id)) return res.status(400).json({ error: `model "${v}" is not a ${requireAdapter(id).label} model` }); patch[key] = v; } + // Validate the resulting model/effort pair together before any settings are written. + const currentSettings = getSettings(); + for (const id of ENGINES) { + const adapter = requireAdapter(id); + const key = adapter.defaultEffortKey; + if (body[key] !== undefined) { + if (typeof body[key] !== "string") return res.status(400).json({ error: `${key} must be a string` }); + patch[key] = body[key].trim().toLowerCase(); + } + if (patch[key] === undefined && patch[adapter.defaultModelKey] === undefined) continue; + const effort = patch[key] ?? currentSettings[key] ?? ""; + const model = patch[adapter.defaultModelKey] ?? currentSettings[adapter.defaultModelKey] ?? ""; + if (effort && !effortBelongsToModel(effort, id, model)) { + return res.status(400).json({ error: `effort "${effort}" is not supported by the selected ${adapter.label} model` }); + } + } // Cross-engine failover. `codexFallback` is the pre-rename key an older UI still sends; both // write the canonical one so the two can never disagree on disk. if (typeof body.engineFallback === "boolean") patch.engineFallback = body.engineFallback; diff --git a/test/default-model.test.js b/test/default-model.test.js index 9ab417c5..b17f8f0a 100644 --- a/test/default-model.test.js +++ b/test/default-model.test.js @@ -7,7 +7,7 @@ import { ensureTestEnv } from "./helpers.js"; ensureTestEnv(); // scratch gateway dir — getSettings() reads no real config -const { getDefaultModel, saveSettings, settingsForApi } = await import("../src/config/settings.js"); +const { getDefaultModel, getDefaultEffort, saveSettings, settingsForApi } = await import("../src/config/settings.js"); const { isValidModel } = await import("../src/slack/util.js"); const { invalidModelOrEffort } = await import("../src/web/routes/helpers.js"); @@ -46,3 +46,20 @@ test("the admin route's isValidModel guard accepts the intended ids and rejects assert.equal(invalidModelOrEffort({ model: "gpt-6-astra", effort: "ultra" }), ""); assert.match(invalidModelOrEffort({ effort: "extreme" }), /effort must be one of/); }); + + +test("default effort is independent per harness, safe for legacy settings, and clearable", async () => { + const { ENGINE_IDS, requireAdapter } = await import("../src/engines/registry.js"); + for (const id of ENGINE_IDS) { + const key = requireAdapter(id).defaultEffortKey; + assert.equal(getDefaultEffort(id), ""); + saveSettings({ [key]: " HIGH " }); + assert.equal(getDefaultEffort(id), "high"); + assert.equal(settingsForApi()[key], "high"); + if (id.startsWith("qwen")) assert.equal(settingsForApi().qwenProviders.find((p) => p.id === id).defaultEffort, "high"); + for (const value of ["unsupported", 42, ""]) { + saveSettings({ [key]: value }); + assert.equal(getDefaultEffort(id), ""); + } + } +}); diff --git a/test/engine-adapter-contract.test.js b/test/engine-adapter-contract.test.js index 99e24c2e..5d477529 100644 --- a/test/engine-adapter-contract.test.js +++ b/test/engine-adapter-contract.test.js @@ -7,7 +7,7 @@ import { engineUiManifest, fallbackTargets, requireAdapter } from "../src/engine function fakeAdapter(id = "third") { return { - id, label: "Third", cli: "third", defaultModelKey: "defaultThirdModel", mcpMetaKey: "allowedThirdMcps", + id, label: "Third", cli: "third", defaultModelKey: "defaultThirdModel", defaultEffortKey: "defaultThirdEffort", mcpMetaKey: "allowedThirdMcps", instructionFile: "THIRD.md", skillsDir: ".third/skills", mcpTransport: "argv", contextWindow: 1000, efforts: ["low"], models: [{ label: "Third One", value: "third-1" }], mintsOwnSessionId: true, supports: { warmPool: false, networkModes: ["off"] }, modelBelongs: (m) => m.startsWith("third-"), diff --git a/test/engine-registry.test.js b/test/engine-registry.test.js index 2712c518..b42c8d74 100644 --- a/test/engine-registry.test.js +++ b/test/engine-registry.test.js @@ -21,7 +21,7 @@ test("all shipped engines are registered with the facts callers depend on", () = for (const id of ENGINE_IDS) { const a = adapterFor(id); assert.equal(a.id, id); - for (const field of ["label", "cli", "defaultModelKey", "mcpMetaKey", "instructionFile", "skillsDir", "mcpTransport"]) { + for (const field of ["label", "cli", "defaultModelKey", "defaultEffortKey", "mcpMetaKey", "instructionFile", "skillsDir", "mcpTransport"]) { assert.ok(a[field], `${id} must declare ${field}`); } assert.equal(typeof a.contextWindow, "number"); diff --git a/test/qwen-provider-settings-browser.test.js b/test/qwen-provider-settings-browser.test.js index fb48602e..1235821a 100644 --- a/test/qwen-provider-settings-browser.test.js +++ b/test/qwen-provider-settings-browser.test.js @@ -16,7 +16,7 @@ ensureTestEnv(); test("each provider gets its own settings card, and a key typed in one is saved under that provider", { skip: !process.env.CG_BROWSER_MODULE }, async (t) => { const { chromium } = await import(process.env.CG_BROWSER_MODULE); const { createAdminRouter } = await import("../src/web/routes/admin.js"); - const { saveSettings, getQwenConfig, getDefaultModel } = await import("../src/config/settings.js"); + const { saveSettings, getQwenConfig, getDefaultModel, getDefaultEffort } = await import("../src/config/settings.js"); const { QWEN_PROVIDERS } = await import("../src/engines/qwen.js"); saveSettings(Object.fromEntries(QWEN_PROVIDERS.flatMap((e) => [[e.settings.apiKey, ""], [e.settings.baseUrl, ""], [e.defaultModelKey, ""]]))); @@ -79,7 +79,18 @@ test("each provider gets its own settings card, and a key typed in one is saved await eu.locator('[data-role="api-key"]').fill("sk-ws-BROWSER-FIXTURE-5678"); await eu.locator('[data-role="base-url"]').fill("https://ws-browser.eu-central-1.example/apps/anthropic"); await eu.locator('[data-role="default-model"]').selectOption(euEntry.models[0].value); + await eu.locator('[data-role="default-effort"]').selectOption("medium"); + await page.locator("#set-default-claude-effort").selectOption("high"); + await page.locator("#set-default-codex-effort").selectOption("ultra"); await persist(); + assert.equal(getDefaultEffort("claude"), "high"); + assert.equal(getDefaultEffort("codex"), "ultra"); + assert.equal(getDefaultEffort(euEntry.id), "medium"); + await page.reload(); + await cards.first().waitFor(); + assert.equal(await page.locator("#set-default-claude-effort").inputValue(), "high"); + assert.equal(await page.locator("#set-default-codex-effort").inputValue(), "ultra"); + assert.equal(await eu.locator('[data-role="default-effort"]').inputValue(), "medium"); assert.equal(getQwenConfig(euEntry.id).apiKey, "sk-ws-BROWSER-FIXTURE-5678", "the key is stored under the provider it was typed into"); assert.equal(getQwenConfig(euEntry.id).baseUrl, "https://ws-browser.eu-central-1.example/apps/anthropic"); @@ -98,6 +109,14 @@ test("each provider gets its own settings card, and a key typed in one is saved assert.equal(getQwenConfig(euEntry.id).apiKey, "", "the armed clear removed it"); assert.equal(await eu.locator('[data-role="key-state"]').textContent(), "not set"); + await page.locator("#set-default-claude-effort").selectOption(""); + await page.locator("#set-default-codex-effort").selectOption(""); + await eu.locator('[data-role="default-effort"]').selectOption(""); + await persist(); + assert.equal(getDefaultEffort("claude"), ""); + assert.equal(getDefaultEffort("codex"), ""); + assert.equal(getDefaultEffort(euEntry.id), ""); + // The settings search indexes each card separately — they are siblings, not one wrapped block. await page.locator("#settings-search").fill("Model Studio"); await page.waitForFunction(() => !globalThis.document.querySelectorAll(".qwen-provider-card")[1]?.classList.contains("filtered-out")); diff --git a/test/qwen-provider-settings-route.test.js b/test/qwen-provider-settings-route.test.js index 81d90619..08297986 100644 --- a/test/qwen-provider-settings-route.test.js +++ b/test/qwen-provider-settings-route.test.js @@ -107,3 +107,52 @@ test("enabled provider cannot become default without its own key and endpoint; c settings.saveSettings({ engine: "claude" }); clearAll(); }); + + +test("gateway effort defaults validate each resulting model/effort pair atomically", async () => { + const { ENGINE_IDS, requireAdapter, effortsForModel } = await import("../src/engines/registry.js"); + for (const id of ENGINE_IDS) { + const adapter = requireAdapter(id); + const key = adapter.defaultEffortKey; + const model = adapter.models[0]?.value || ""; + const effort = effortsForModel(id, model)[0]; + const saved = await put({ [adapter.defaultModelKey]: model, [key]: ` ${effort.toUpperCase()} ` }); + assert.equal(settings.getDefaultEffort(id), effort); + assert.equal(saved[key], effort); + await put({ [key]: 42 }, 400); + await put({ [key]: "unsupported", [adapter.defaultModelKey]: "" }, 400); + assert.equal(settings.getDefaultEffort(id), effort, "rejected requests do not modify either setting"); + assert.equal(settings.getDefaultModel(id), model); + await put({ [key]: "" }); + assert.equal(settings.getDefaultEffort(id), ""); + } + // Codex's expanded reasoning levels cannot leak into a Claude default. + await put({ defaultClaudeEffort: "ultra" }, 400); +}); + + +test("effort validation follows the selected model, including a model-only change", async () => { + const { mkdirSync, writeFileSync } = await import("node:fs"); + const path = await import("node:path"); + const { refreshEngineModels } = await import("../src/engines/registry.js"); + const { gatewayRoot } = await import("../src/config/paths.js"); + const bin = path.join(gatewayRoot(), "discovery-fixture-bin"); + mkdirSync(bin, { recursive: true }); + const catalog = { models: [ + { slug: "gpt-effort-wide", visibility: "list", supported_reasoning_levels: [{ effort: "low" }, { effort: "high" }] }, + { slug: "gpt-effort-narrow", visibility: "list", supported_reasoning_levels: [{ effort: "low" }] }, + ] }; + writeFileSync(path.join(bin, "codex"), `#!/usr/bin/env node\nprocess.stdout.write(${JSON.stringify(JSON.stringify(catalog))});\n`, { mode: 0o700 }); + const originalPath = process.env.PATH; + try { + process.env.PATH = `${bin}${path.delimiter}${originalPath}`; + await refreshEngineModels("codex", { force: true }); + } finally { process.env.PATH = originalPath; } + await put({ defaultCodexModel: "gpt-effort-wide", defaultCodexEffort: "high" }); + await put({ defaultCodexModel: "gpt-effort-narrow" }, 400); + assert.equal(settings.getDefaultModel("codex"), "gpt-effort-wide"); + await put({ defaultCodexModel: "gpt-effort-narrow", defaultCodexEffort: "high" }, 400); + await put({ defaultCodexModel: "gpt-effort-narrow", defaultCodexEffort: "low" }); + assert.equal(settings.getDefaultModel("codex"), "gpt-effort-narrow"); + assert.equal(settings.getDefaultEffort("codex"), "low"); +}); diff --git a/test/runtime-identity-preamble.test.js b/test/runtime-identity-preamble.test.js index 148da8d4..c2e37990 100644 --- a/test/runtime-identity-preamble.test.js +++ b/test/runtime-identity-preamble.test.js @@ -24,11 +24,11 @@ backend.prepareTarget = (base) => { await useFakeRuntime(backend); const { setUser, upsertChannelEntry, saveChannelMeta, patchChannelMeta } = await import("../src/config/store.js"); const { saveSettings } = await import("../src/config/settings.js"); -const { setThreadModel, setThreadClean } = await import("../src/gateway/thread-engine.js"); +const { setThreadModel, setThreadEffort, setThreadClean } = await import("../src/gateway/thread-engine.js"); const { runMessage } = await import("../src/gateway/run.js"); async function fixture(name, engine, model, extra = {}) { - saveSettings({ engine, defaultClaudeModel: "sonnet", defaultCodexModel: "gpt-5.6-sol", engineFallback: true, + saveSettings({ defaultClaudeEffort: "", defaultCodexEffort: "", engine, defaultClaudeModel: "sonnet", defaultCodexModel: "gpt-5.6-sol", engineFallback: true, engineEnabled: { claude: true, codex: true }, agentMemory: false, memoryReviewEvery: 0, composioMode: "personal", containerFullAccessHome: false }); const authorId = `U_${name}`; const channelId = `C_${name}`; @@ -310,3 +310,49 @@ test("a lost resumed session gets fresh metadata when the gateway heals it", asy assert.equal(metadata(calls[0]).session, "fresh"); assert.ok(calls[0].args.some((arg) => String(arg).includes("Clean mode for this attempt: **enabled**"))); }); + + +for (const engine of ["claude", "codex"]) { + test(`${engine}: default effort reaches fresh/resumed flags and respects every override layer`, async () => { + const context = await fixture(`DEFAULT_EFFORT_${engine}`, engine, "", { effort: "" }); + const key = engine === "claude" ? "defaultClaudeEffort" : "defaultCodexEffort"; + saveSettings({ [key]: "low" }); + const start = backend.calls.spawn.length; + await runMessage({ ...context, text: "Report your configured effort." }); + saveSettings({ [key]: "high" }); + await runMessage({ ...context, text: "And now?" }); + await patchChannelMeta(context.entry.slug, { effort: "medium" }); + await runMessage({ ...context, text: "Use the channel setting." }); + await setThreadEffort(context.entry.slug, context.threadKey, "low"); + await runMessage({ ...context, text: "Use the thread setting." }); + await runMessage({ ...context, text: "Use this request setting.", overrides: { effort: "high" } }); + await setThreadEffort(context.entry.slug, context.threadKey, ""); + await patchChannelMeta(context.entry.slug, { effort: "" }); + saveSettings({ [key]: "" }); + await runMessage({ ...context, text: "Use the CLI default." }); + const calls = attempts(start); + assert.equal(calls.length, 6); + const expected = ["low", "high", "medium", "low", "high", null]; + assert.deepEqual(calls.map((call) => metadata(call).configured_effort), expected); + for (let i = 0; i < calls.length; i++) { + const call = calls[i]; + if (engine === "claude") { + assert.equal(call.args.includes("--effort"), Boolean(expected[i])); + if (expected[i]) assert.equal(call.args[call.args.indexOf("--effort") + 1], expected[i]); + } else { + const flag = call.args.find((arg) => String(arg).startsWith("model_reasoning_effort=")); + assert.equal(flag, expected[i] ? `model_reasoning_effort="${expected[i]}"` : undefined); + } + } + assert.equal(metadata(calls[1]).session, "resumed"); + }); +} + +test("failover and its model retry use the destination harness default effort", async () => { + const context = await fixture("DEFAULT_EFFORT_FAILOVER", "claude", "gpt-5.6", { effort: "medium", cleanMode: true }); + saveSettings({ defaultClaudeEffort: "low", defaultCodexEffort: "xhigh" }); + const start = backend.calls.spawn.length; + await runMessage({ ...context, text: "CLAUDE_STUB_LIMIT_FAIL_SAFE CODEX_STUB_REJECT_MODEL" }); + const calls = attempts(start); + assert.deepEqual(calls.map((call) => metadata(call).configured_effort), ["medium", "xhigh", "xhigh"]); +});