From aea3ce90d7e03868973d00eb7e57c1850f892930 Mon Sep 17 00:00:00 2001 From: Chris Phillipson Date: Fri, 31 Jul 2026 02:20:44 -0700 Subject: [PATCH] fix(live): graded inference-provider provenance for Claude and Codex sessions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every Live session rendered "Provider not established". Two causes, fixed per ADR-0021: - Codex records its serving provider as model_provider (rollout session_meta and the state ledger threads table); the adapters read a bare provider field that does not exist, silently dropping observed evidence including custom model_providers entries (openrouter/azure/ollama). Read model_provider with legacy tolerance. - Claude Code transcripts never name the serving endpoint. Resolve it from the host's documented selection surface (CLAUDE_CODE_USE_BEDROCK/_VERTEX/ _FOUNDRY, ANTHROPIC_BASE_URL gateway classification) across settings layers (managed > project local > shared > user > shell, empty string = unset), yielding configured provenance for explicit selections and inferred for the first-party default — never observed. Supporting invariants: projection retains established provider/model claims when later events carry none (server + browser reducers); metadata learned during transcript bootstrap (codex session_meta identity) persists into live tailing; the UI names bedrock/vertex/foundry/gateway and falls back to subagent-rooted nodes for child session cards. --- docs/LIVE-SESSIONS.md | 6 +- .../adr/0021-inference-provider-provenance.md | 67 +++++++++++++++ docs/adr/README.md | 7 ++ docs/ddd/live-sessions.md | 8 ++ src/lib/codex-state.mjs | 7 +- src/lib/dashboard/live/client.mjs | 8 +- src/lib/live/claude-adapter.mjs | 6 +- src/lib/live/claude-provider.mjs | 78 +++++++++++++++++ src/lib/live/codex-adapter.mjs | 7 +- src/lib/live/live-sessions-service.mjs | 27 +++++- src/lib/live/projection.mjs | 7 ++ tests/kit/codex-state.test.mjs | 18 +++- tests/kit/live-adapters.test.mjs | 37 ++++++++ tests/kit/live-claude-provider.test.mjs | 86 +++++++++++++++++++ tests/kit/live-core.test.mjs | 20 +++++ tests/kit/live-service.test.mjs | 53 ++++++++++++ 16 files changed, 428 insertions(+), 14 deletions(-) create mode 100644 docs/adr/0021-inference-provider-provenance.md create mode 100644 src/lib/live/claude-provider.mjs create mode 100644 tests/kit/live-claude-provider.test.mjs diff --git a/docs/LIVE-SESSIONS.md b/docs/LIVE-SESSIONS.md index 8706282..ab2f02c 100644 --- a/docs/LIVE-SESSIONS.md +++ b/docs/LIVE-SESSIONS.md @@ -35,7 +35,11 @@ request. Stopping the dashboard closes the live service and all clients. stores and bootstrapped from bounded, metadata-only records. - Project-first session cards with a host glyph and name, independently evidenced inference provider/model when reported, lifecycle, freshness, and - a concise operational summary. + a concise operational summary. Provider identity is graded (ADR-0021): + observed from Codex artifacts (`model_provider`), configured/inferred from + Claude Code's documented selection surface (Bedrock, Vertex, Foundry, + `ANTHROPIC_BASE_URL` gateways such as OpenRouter), and "Provider not + established" only when no evidence of any grade exists. - Session, agent, sub-agent, tool, skill, plugin, MCP, and gate entities when a supported source record identifies them. - Authoritative Codex parent/child edges from the Codex state ledger. diff --git a/docs/adr/0021-inference-provider-provenance.md b/docs/adr/0021-inference-provider-provenance.md new file mode 100644 index 0000000..d022936 --- /dev/null +++ b/docs/adr/0021-inference-provider-provenance.md @@ -0,0 +1,67 @@ +# ADR-0021 — Inference-provider provenance for live sessions + +- **Status:** Accepted +- **Date:** 2026-07-31 +- **Deciders:** agentic-kit maintainers +- **Related:** [ADR-0012](0012-live-sessions-observability.md) + +## Context + +ADR-0012 keeps the execution-host and inference-provider axes independent and forbids promoting +a host-based assumption to an observed provider claim. In practice every Claude Code session in +the Live view rendered "Provider not established", and — unnoticed — every Codex session did too. +Investigation against current upstream sources found two distinct causes: + +1. **Codex records the provider; ak read the wrong field.** Codex rollout `session_meta` + payloads and the `threads` table of `state_N.sqlite` both spell the field `model_provider` + (verified live against `~/.codex` and openai/codex session examples). The adapters and the + state reader looked for a bare `provider` column/key that does not exist, so real observed + evidence — including custom `model_providers` entries such as `openrouter`, `azure`, or + `ollama` from `~/.codex/config.toml` — was silently dropped by the schema-tolerant column + filter. +2. **Claude Code genuinely never writes the provider.** No record type in + `~/.claude/projects/*/*.jsonl` carries one. Which endpoint serves a session is decided by the + host's documented configuration surface instead: `CLAUDE_CODE_USE_BEDROCK`, + `CLAUDE_CODE_USE_VERTEX`, `CLAUDE_CODE_USE_FOUNDRY`, and `ANTHROPIC_BASE_URL` (gateway/proxy), + where settings-file `env` blocks override the shell with managed > project local > project + shared > user precedence and an empty string means "unset at this layer" + (code.claude.com/docs/en/env-vars and the Bedrock/Vertex/Foundry deployment guides). + +The event schema already reserved the vocabulary for this: `providerProvenance ∈ observed | +configured | inferred | unknown`, populated from field-level evidence. Only emitters were missing. + +## Decision + +- **Codex: read `model_provider`** in the rollout adapter and the state-ledger reader, keeping + the bare `provider` spelling only as legacy tolerance. The claim remains **observed** — it is + in the artifact. Custom provider ids pass through untranslated. +- **Claude: resolve the provider from the host's configuration surface** at transcript-discovery + time (`src/lib/live/claude-provider.mjs`), mirroring the documented selection order: + Bedrock flag → Vertex flag → Foundry flag → `ANTHROPIC_BASE_URL` (classified to `anthropic`, + `openrouter`, or generic `gateway` by hostname) → first-party default `anthropic`. Explicit + selections are **configured**; the first-party default is **inferred**; an unknown gateway stays + a `gateway`, never a guessed vendor. Resolution is memoized per session `cwd` and injectable + for tests. +- **Provenance is carried, never upgraded.** Adapters put the resolution's own provenance into + `source.fields.provider`; the schema lifts it into `providerProvenance`. Nothing configured or + inferred may be displayed as observed, per ADR-0012. +- **Identity evidence is retained.** Projection (server and browser reducers alike) keeps an + established provider/model claim when a later event for the same node carries none — absence of + evidence on one event does not erase evidence already observed. Metadata learned while + bootstrapping a transcript (codex `session_meta` identity in particular) persists into live + tailing for the same reason. +- **Presentation** names the new values (AWS Bedrock, Google Vertex AI, Microsoft Foundry, + Custom gateway) and continues to render "Provider not established" only when no evidence of any + grade exists. + +## Consequences + +- Codex sessions now show their serving provider with observed provenance, including + OpenRouter/Azure/Ollama-served sessions configured through `model_providers`. +- Claude Code sessions show who serves Claude models — Anthropic, Bedrock, Vertex, Foundry, + OpenRouter, or an unnamed gateway — labeled configured or inferred, answering the operator + question without overclaiming. +- OpenCode is unaffected: it is not a live host (ADR-0017/0018 scope it to managed execution), + and its usage-side records already carry an observed `providerID`. +- If codex renames or Claude Code starts recording the serving endpoint, the adapters prefer the + in-artifact (observed) value automatically. diff --git a/docs/adr/README.md b/docs/adr/README.md index 68aa882..6232928 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -29,6 +29,7 @@ Consequences**, and cites the grounded source it rests on where relevant. | [0018](0018-generalized-host-worker-execution.md) | Generalized host-worker execution; `ak run` canonical | Accepted; compatibility amended | | [0019](0019-escalation-in-ak-run.md) | Bounded per-worker escalation in `ak run` | Accepted; historical context closed | | [0020](0020-ga-stable-surfaces.md) | One stable GA surface per capability | Implemented | +| [0021](0021-inference-provider-provenance.md) | Inference-provider provenance for live sessions | Accepted | Theme: ADRs **0001–0006** define **dual-host LLM routing and leadership** — how `ak` lets ruflo route each development activity (architecture, implementation, testing, review, …) to the right host (Claude @@ -99,3 +100,9 @@ clauses are superseded by ADR-0020. host-management namespace, and the versioned top-level routing envelope; removes dynamic adapter-specific execution bootstrap; limits old vocabulary to marked decision history and one upgrade section; and preserves OpenCode's opt-in, supervised, non-primary, non-AQE boundary. + +**0021** makes the Live view answer who serves each session's models without overclaiming. Codex's +in-artifact `model_provider` (rollouts + state ledger) is read as observed evidence; Claude Code's +provider is resolved from its documented configuration surface (Bedrock/Vertex/Foundry flags, +`ANTHROPIC_BASE_URL` gateways) with configured/inferred provenance, since its transcripts never +record the serving endpoint. diff --git a/docs/ddd/live-sessions.md b/docs/ddd/live-sessions.md index f91c920..4adb0bd 100644 --- a/docs/ddd/live-sessions.md +++ b/docs/ddd/live-sessions.md @@ -219,6 +219,14 @@ multiple relationship types. Project, provider, model, role, lifecycle, and hierarchy each retain their own provenance. A host may suggest a provider, but that assumption cannot be displayed as an observed provider fact. +Provider evidence is graded by where it came from (ADR-0021). Codex artifacts name their serving +provider (`model_provider` in rollout `session_meta` and the state ledger), so Codex claims are +**observed**. Claude transcripts never name one; the domain resolves it from the host's documented +configuration surface — Bedrock/Vertex/Foundry selection flags and `ANTHROPIC_BASE_URL` gateway +classification across settings layers — yielding a **configured** claim, or an **inferred** one for +the first-party default. An unrecognized gateway stays `gateway` rather than a guessed vendor, and +a claim's grade is never upgraded downstream. + ### `Activity` An activity represents a bounded invocation or evaluation. It has source-scoped identity, actor, diff --git a/src/lib/codex-state.mjs b/src/lib/codex-state.mjs index 4162a01..d6215d7 100644 --- a/src/lib/codex-state.mjs +++ b/src/lib/codex-state.mjs @@ -62,7 +62,7 @@ export function readCodexState(opts = {}) { const pick = ['id', 'thread_source'] .concat([ 'tokens_used', 'source', 'model', 'git_branch', 'agent_nickname', 'agent_role', - 'cwd', 'provider', 'status', + 'cwd', 'model_provider', 'provider', 'status', ].filter((c) => cols.has(c))); const threads = new Map(); for (const row of db.prepare(`SELECT ${pick.join(', ')} FROM threads`).all()) { @@ -75,7 +75,10 @@ export function readCodexState(opts = {}) { agentNickname: typeof row.agent_nickname === 'string' ? row.agent_nickname : null, agentRole: typeof row.agent_role === 'string' ? row.agent_role : null, project: typeof row.cwd === 'string' ? resolveProjectLabel(row.cwd) : null, - provider: typeof row.provider === 'string' ? row.provider : null, + // codex's threads schema names this model_provider; the bare provider + // spelling is kept for tolerance of older/forked ledgers. + provider: typeof row.model_provider === 'string' ? row.model_provider + : (typeof row.provider === 'string' ? row.provider : null), status: typeof row.status === 'string' ? row.status : null, }); } diff --git a/src/lib/dashboard/live/client.mjs b/src/lib/dashboard/live/client.mjs index 16261d5..8e8c6be 100644 --- a/src/lib/dashboard/live/client.mjs +++ b/src/lib/dashboard/live/client.mjs @@ -28,9 +28,9 @@ export const LIVE_JS = ` function hostOf(v){return String(v&&v.host||"internal").toLowerCase();} function hostName(v){var h=hostOf(v);return{claude:"Claude Code",codex:"Codex",internal:"Internal"}[h]||title(h);} function hostGlyph(v){return{claude:"✳",codex:"◉",internal:"⌁"}[hostOf(v)]||"⌁";} - function inferenceProviderOf(v){if(v&&v.provider)return String(v.provider);var nodes=v&&v.nodes||[],session=nodes.find(function(n){return n.kind==="session"&&n.provider;});return session&&session.provider||null;} - function inferenceProviderName(v){var p=inferenceProviderOf(v);if(!p)return"Provider not established";return{anthropic:"Anthropic",openai:"OpenAI",openrouter:"OpenRouter",ollama:"Ollama",google:"Google Gemini"}[String(p).toLowerCase()]||title(p);} - function providerEvidence(v){var p=v&&v.providerProvenance;if(!p&&v&&v.nodes){var n=v.nodes.find(function(x){return x.kind==="session"&&x.provider;});p=n&&n.providerProvenance;}return p||"unknown";} + function inferenceProviderOf(v){if(v&&v.provider)return String(v.provider);var nodes=v&&v.nodes||[],session=nodes.find(function(n){return n.kind==="session"&&n.provider;})||nodes.find(function(n){return n.kind==="subagent"&&n.provider;});return session&&session.provider||null;} + function inferenceProviderName(v){var p=inferenceProviderOf(v);if(!p)return"Provider not established";return{anthropic:"Anthropic",openai:"OpenAI",openrouter:"OpenRouter",ollama:"Ollama",google:"Google Gemini",bedrock:"AWS Bedrock",vertex:"Google Vertex AI",foundry:"Microsoft Foundry",gateway:"Custom gateway"}[String(p).toLowerCase()]||title(p);} + function providerEvidence(v){var p=v&&v.providerProvenance;if(!p&&v&&v.nodes){var n=v.nodes.find(function(x){return x.kind==="session"&&x.provider;})||v.nodes.find(function(x){return x.kind==="subagent"&&x.provider;});p=n&&n.providerProvenance;}return p||"unknown";} function kindName(k){return{session:"Coordinator",agent:"Agent",subagent:"Worker",tool:"Tool",skill:"Skill",plugin:"Plugin",mcp:"MCP tool",gate:"Quality gate"}[k]||title(k||"agent");} function statusText(v){return{running:"Working",queued:"Queued",quiescent:"Waiting",expired:"No recent activity",blocked:"Blocked",completed:"Completed",failed:"Failed",cancelled:"Cancelled",idle:"Waiting"}[v]||"Outcome unavailable";} function label(n){return n&&(n.toolName||n.label||n.toolCategory||n.role&&title(n.role)||kindName(n.kind))||"Activity";} @@ -52,7 +52,7 @@ export const LIVE_JS = ` if(!ev||!ev.sessionId||!ev.actor||!ev.actor.id)return;if(ev.eventId&&state.seen[ev.eventId])return; var sessions=state.snapshot.sessions.slice(),evKey=String(ev.sessionKey||ev.host+":"+ev.sessionId),i=sessions.findIndex(function(s){return sessionKey(s)===evKey;}),s=i>=0?Object.assign({},sessions[i]):{id:ev.sessionId,key:evKey,project:ev.project,projectKey:ev.projectKey,host:ev.host,status:"unknown",nodes:[],edges:[]};if(ev.parentSessionId)s.parentSessionId=ev.parentSessionId; s.nodes=(s.nodes||[]).slice();s.edges=(s.edges||[]).slice();var targetTerminal=!!ev.target&&/\\.(completed|failed|cancelled)$/.test(ev.action||""),priorActor=s.nodes.find(function(n){return n.id===ev.actor.id;}); - function upsert(a,status){var at=s.nodes.findIndex(function(n){return n.id===a.id;}),old=at>=0?s.nodes[at]:{},provider=a.provider||old.provider||null,next=Object.assign({},old,{id:a.id,kind:a.kind||old.kind,role:a.role||old.role,provider:provider,providerProvenance:provider?(ev.providerProvenance||old.providerProvenance||"unknown"):"unknown",model:a.model||old.model,label:a.label||old.label,toolName:a.toolName||old.toolName,host:ev.host||old.host,surface:ev.surface||old.surface,status:status||old.status,observedAt:ev.observedAt,confidence:ev.source&&ev.source.confidence||old.confidence,lastAction:ev.action,durationMs:ev.attributes&&ev.attributes.durationMs||old.durationMs});if(TERMINAL[old.status]&&!TERMINAL[next.status])next.status=old.status;if(at>=0)s.nodes[at]=next;else s.nodes.push(next);} + function upsert(a,status){var at=s.nodes.findIndex(function(n){return n.id===a.id;}),old=at>=0?s.nodes[at]:{},provider=a.provider||old.provider||null,next=Object.assign({},old,{id:a.id,kind:a.kind||old.kind,role:a.role||old.role,provider:provider,providerProvenance:provider?(a.provider?(ev.providerProvenance||"unknown"):(old.providerProvenance||ev.providerProvenance||"unknown")):"unknown",model:a.model||old.model,label:a.label||old.label,toolName:a.toolName||old.toolName,host:ev.host||old.host,surface:ev.surface||old.surface,status:status||old.status,observedAt:ev.observedAt,confidence:ev.source&&ev.source.confidence||old.confidence,lastAction:ev.action,durationMs:ev.attributes&&ev.attributes.durationMs||old.durationMs});if(TERMINAL[old.status]&&!TERMINAL[next.status])next.status=old.status;if(at>=0)s.nodes[at]=next;else s.nodes.push(next);} upsert(ev.actor,targetTerminal?priorActor&&priorActor.status||"running":ev.status);if(ev.target&&ev.target.id){upsert(Object.assign({},ev.target,{label:ev.attributes&&(ev.attributes.toolName||ev.attributes.toolCategory)||ev.target.label,toolName:ev.attributes&&ev.attributes.toolName}),targetTerminal?ev.status:"running");var eid=ev.actor.id+"|"+ev.action+"|"+ev.target.id,edge={id:eid,source:ev.actor.id,target:ev.target.id,action:ev.action,confidence:ev.source&&ev.source.confidence},ei=s.edges.findIndex(function(x){return x.id===eid;});if(ei>=0)s.edges[ei]=edge;else s.edges.push(edge);} if(ev.actor.kind==="session"&&/^session\\./.test(ev.action||""))s.status=ev.status;if(ev.status==="running"&&(!ev.source||ev.source.adapter!=="codex-state"))s.lifecycle="active";else if(!s.lifecycle)s.lifecycle="historical";s.updatedAt=ev.observedAt;if(i>=0)sessions[i]=s;else sessions.push(s); state.snapshot={schemaVersion:state.snapshot.schemaVersion||1,cursor:ev.eventId||state.snapshot.cursor,projects:state.snapshot.projects,sessions:sessions,health:state.snapshot.health};if(ev.eventId)state.seen[ev.eventId]=1;state.events.unshift(ev);state.events=state.events.slice(0,100);state.lastAt=Date.now();if(!state.selected)selectSession(sessionKey(s),false);if(state.playback.mode==="live")render(); diff --git a/src/lib/live/claude-adapter.mjs b/src/lib/live/claude-adapter.mjs index bb0a13a..231a0d0 100644 --- a/src/lib/live/claude-adapter.mjs +++ b/src/lib/live/claude-adapter.mjs @@ -26,7 +26,11 @@ export function adaptClaudeRecord(record, context = {}) { confidence: isSidechain && !record.agentId ? 'inferred' : 'observed', fields: { project: context.project ? 'observed' : null, - provider: record.provider || context.provider ? 'observed' : null, + // A provider on the record itself is observed evidence; one resolved + // from the host's configuration surface carries that resolution's own + // provenance (configured/inferred) and must not be upgraded. + provider: record.provider ? 'observed' + : (context.provider ? context.providerProvenance ?? 'configured' : null), model: record.message?.model || context.model ? 'observed' : null, status: 'observed', hierarchy: isSidechain ? (record.agentId ? 'observed' : 'inferred') : 'observed', diff --git a/src/lib/live/claude-provider.mjs b/src/lib/live/claude-provider.mjs new file mode 100644 index 0000000..c5c9efe --- /dev/null +++ b/src/lib/live/claude-provider.mjs @@ -0,0 +1,78 @@ +import os from 'node:os'; +import path from 'node:path'; +import { readJson } from '../settings.mjs'; + +// Which endpoint serves a Claude Code session. Claude transcripts never record +// this, so the resolver mirrors the host's documented selection surface +// (code.claude.com/docs/en/env-vars + the Bedrock/Vertex/Foundry guides): +// CLAUDE_CODE_USE_BEDROCK / _VERTEX / _FOUNDRY switch the serving platform and +// ANTHROPIC_BASE_URL points the first-party path at a proxy or gateway. +// Settings `env` blocks override the shell — managed > project local > +// project shared > user — and an empty string means "unset at this layer". +// The result is a claim about configuration, so provenance is 'configured' +// (explicit selection found) or 'inferred' (first-party default), never +// 'observed'. + +/** @type {Record} */ +const MANAGED_SETTINGS = { + darwin: '/Library/Application Support/ClaudeCode/managed-settings.json', + linux: '/etc/claude-code/managed-settings.json', + win32: 'C:\\ProgramData\\ClaudeCode\\managed-settings.json', +}; + +const plainEnv = (value) => value && typeof value === 'object' && !Array.isArray(value) + ? value : null; + +// "1" is the documented spelling; tolerate other truthy strings but never +// treat an explicit "0"/"false" as a selection. +const flag = (value) => typeof value === 'string' && value !== '' + && !['0', 'false'].includes(value.toLowerCase()); + +function gatewayProvider(baseUrl) { + let host = null; + try { host = new URL(baseUrl).hostname.toLowerCase(); } catch { /* unparseable → generic */ } + if (host === 'api.anthropic.com' || host?.endsWith('.anthropic.com')) return 'anthropic'; + if (host?.includes('openrouter')) return 'openrouter'; + return 'gateway'; +} + +/** + * Resolve the inference provider configured for Claude Code sessions rooted at + * `cwd`. Returns { provider, provenance } — provenance 'configured' when an + * explicit selection was found, 'inferred' for the first-party default. + * + * @param {{ cwd?: string, env?: Record, home?: string, + * platform?: string, read?: (file: string) => any }} [opts] test seams + */ +export function resolveClaudeProvider({ + cwd, + env = process.env, + home = os.homedir(), + platform = process.platform, + read = readJson, +} = {}) { + const layers = []; + const managed = MANAGED_SETTINGS[platform]; + if (managed) layers.push(plainEnv(read(managed)?.env)); + if (typeof cwd === 'string' && cwd) { + layers.push(plainEnv(read(path.join(cwd, '.claude', 'settings.local.json'))?.env)); + layers.push(plainEnv(read(path.join(cwd, '.claude', 'settings.json'))?.env)); + } + layers.push(plainEnv(read(path.join(home, '.claude', 'settings.json'))?.env)); + const lookup = (name) => { + for (const layer of layers) { + if (layer && Object.hasOwn(layer, name)) { + const value = layer[name]; + return typeof value === 'string' && value !== '' ? value : null; + } + } + const value = env[name]; + return typeof value === 'string' && value !== '' ? value : null; + }; + if (flag(lookup('CLAUDE_CODE_USE_BEDROCK'))) return { provider: 'bedrock', provenance: 'configured' }; + if (flag(lookup('CLAUDE_CODE_USE_VERTEX'))) return { provider: 'vertex', provenance: 'configured' }; + if (flag(lookup('CLAUDE_CODE_USE_FOUNDRY'))) return { provider: 'foundry', provenance: 'configured' }; + const base = lookup('ANTHROPIC_BASE_URL'); + if (base) return { provider: gatewayProvider(base), provenance: 'configured' }; + return { provider: 'anthropic', provenance: 'inferred' }; +} diff --git a/src/lib/live/codex-adapter.mjs b/src/lib/live/codex-adapter.mjs index dd4fcf7..6d61a57 100644 --- a/src/lib/live/codex-adapter.mjs +++ b/src/lib/live/codex-adapter.mjs @@ -8,6 +8,9 @@ export function adaptCodexRecord(record, context = {}) { if (!record || typeof record !== 'object') return []; const payload = record.payload && typeof record.payload === 'object' ? record.payload : {}; const meta = record.type === 'session_meta' ? payload : context.meta ?? {}; + // codex spells this model_provider in rollout session_meta; bare provider is + // legacy tolerance only. + const metaProvider = meta.model_provider ?? meta.provider; const sessionId = meta.id ?? context.sessionId; if (typeof sessionId !== 'string' || !sessionId) return []; const subagent = meta.thread_source === 'subagent' || context.threadSource === 'subagent'; @@ -19,7 +22,7 @@ export function adaptCodexRecord(record, context = {}) { id: sessionId, kind: subagent ? 'subagent' : 'session', label: meta.agent_nickname ?? context.agentNickname, role: meta.agent_role ?? context.agentRole ?? (subagent ? 'worker' : 'primary'), - provider: meta.provider ?? context.provider, + provider: metaProvider ?? context.provider, model: record.type === 'turn_context' ? payload.model ?? context.model : meta.model ?? context.model, }, @@ -28,7 +31,7 @@ export function adaptCodexRecord(record, context = {}) { confidence: subagent && meta.thread_source !== 'subagent' ? 'correlated' : 'observed', fields: { project: context.project ? 'observed' : null, - provider: meta.provider || context.provider ? 'observed' : null, + provider: metaProvider || context.provider ? 'observed' : null, model: (record.type === 'turn_context' ? payload.model : meta.model) || context.model ? 'observed' : null, status: 'observed', diff --git a/src/lib/live/live-sessions-service.mjs b/src/lib/live/live-sessions-service.mjs index fca4859..5bdb0a7 100644 --- a/src/lib/live/live-sessions-service.mjs +++ b/src/lib/live/live-sessions-service.mjs @@ -3,6 +3,7 @@ import path from 'node:path'; import { claudeDir, codexDir } from '../paths.mjs'; import { readCodexState as defaultReadCodexState } from '../codex-state.mjs'; import { adaptClaudeRecord } from './claude-adapter.mjs'; +import { resolveClaudeProvider as defaultResolveClaudeProvider } from './claude-provider.mjs'; import { adaptCodexLedger, adaptCodexRecord } from './codex-adapter.mjs'; import { JsonlTailer } from './jsonl-tailer.mjs'; import { @@ -84,6 +85,7 @@ export class LiveSessionsService { #started = false; #edgeKeys = new Set(); #health = new Map(); + #claudeProviders = new Map(); constructor(options = {}) { const roots = options.roots ?? {}; @@ -97,6 +99,7 @@ export class LiveSessionsService { maxFiles: options.maxFiles ?? 256, replayCapacity: options.replayCapacity ?? 2000, readCodexState: options.readCodexState ?? defaultReadCodexState, + resolveClaudeProvider: options.resolveClaudeProvider ?? defaultResolveClaudeProvider, setInterval: options.setInterval ?? globalThis.setInterval, clearInterval: options.clearInterval ?? globalThis.clearInterval, now: options.now ?? (() => new Date().toISOString()), @@ -203,9 +206,14 @@ export class LiveSessionsService { #add(file, context, initial) { if (this.#tailers.has(file) || this.#tailers.size >= this.#options.maxFiles) return; if (initial && ['claude', 'codex'].includes(context.adapter)) { + // One shared bootstrap context so metadata learned early (codex + // session_meta id/meta, project, model, provider) persists across the + // replayed records and into live tailing below. + const bootstrap = { ...context, bootstrap: true }; for (const record of bootstrapRecords(file, context.adapter)) { - this.#record(record, { ...context, bootstrap: true }, file); + this.#record(record, bootstrap, file); } + Object.assign(context, bootstrap, { bootstrap: false }); } const onRecord = (record) => this.#record(record, context, file); const onError = (error) => this.#error(context.adapter, error); @@ -219,6 +227,13 @@ export class LiveSessionsService { const explicitCwd = record?.cwd ?? (['session_meta', 'turn_context'].includes(record?.type) ? record.payload?.cwd : null); if (explicitCwd) context.project = resolveProjectLabel(explicitCwd); + if (context.adapter === 'claude' && explicitCwd && !context.provider) { + const resolved = this.#claudeProvider(explicitCwd); + if (resolved?.provider) { + context.provider = resolved.provider; + context.providerProvenance = resolved.provenance; + } + } const explicitModel = record?.message?.model ?? (['session_meta', 'turn_context'].includes(record?.type) ? record.payload?.model : null); if (typeof explicitModel === 'string') context.model = explicitModel; @@ -243,6 +258,16 @@ export class LiveSessionsService { for (const event of events) this.#publish(event, context.adapter); } + /** Configuration reads are per-project, so memoize by session cwd. */ + #claudeProvider(cwd) { + if (!this.#claudeProviders.has(cwd)) { + let resolved = null; + try { resolved = this.#options.resolveClaudeProvider({ cwd }); } catch { /* stays unresolved */ } + this.#claudeProviders.set(cwd, resolved); + } + return this.#claudeProviders.get(cwd); + } + #publish(event, adapter) { const published = this.#stream.publish(event); this.#projection = reduceLiveEvent(this.#projection, published, { diff --git a/src/lib/live/projection.mjs b/src/lib/live/projection.mjs index a9859d9..d208a45 100644 --- a/src/lib/live/projection.mjs +++ b/src/lib/live/projection.mjs @@ -46,6 +46,13 @@ export function reduceLiveEvent(projection, event, { if (!actorNode.role || (actorNode.role === 'worker' && priorNode.role !== 'worker')) { actorNode.role = priorNode.role; } + // Identity evidence is retained: an event that carries no provider/model + // claim is absence of evidence, not evidence the earlier claim is gone. + if (!actorNode.provider && priorNode.provider) { + actorNode.provider = priorNode.provider; + actorNode.providerProvenance = priorNode.providerProvenance; + } + actorNode.model ??= priorNode.model; } if (event.target && event.action.endsWith('.completed')) { actorNode.status = priorNode?.status ?? 'running'; diff --git a/tests/kit/codex-state.test.mjs b/tests/kit/codex-state.test.mjs index 7e53938..076a134 100644 --- a/tests/kit/codex-state.test.mjs +++ b/tests/kit/codex-state.test.mjs @@ -13,15 +13,16 @@ import { applyCodexLedger } from '../../src/lib/usage-index.mjs'; const tmp = () => fs.mkdtempSync(path.join(os.tmpdir(), 'ak-codex-state-')); /** Build a fixture ledger shaped like the real state_5.sqlite (verified live - * 2026-07-27: threads has id/thread_source/source/model/git_branch/tokens_used - * among 32 columns; thread_spawn_edges has parent/child/status). */ + * 2026-07-31: threads has id/thread_source/source/model/git_branch/tokens_used + * and model_provider — NOT a bare `provider` column — among 33 columns; + * thread_spawn_edges has parent/child/status). */ function fixtureDb(dir, name = 'state_5.sqlite') { const file = path.join(dir, name); const db = new DatabaseSync(file); db.exec(`CREATE TABLE threads ( id TEXT PRIMARY KEY, thread_source TEXT, source TEXT, model TEXT, git_branch TEXT, tokens_used INTEGER, agent_nickname TEXT, agent_role TEXT, - title TEXT, name TEXT, cwd TEXT, provider TEXT, status TEXT);`); + title TEXT, name TEXT, cwd TEXT, model_provider TEXT, status TEXT);`); db.exec(`CREATE TABLE thread_spawn_edges ( parent_thread_id TEXT, child_thread_id TEXT, status TEXT);`); db.prepare('INSERT INTO threads VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)') @@ -64,6 +65,17 @@ test('readCodexState reads threads and spawn edges', () => { assert.equal(ledger.parents.get('child-1'), 'parent-1'); }); +test('readCodexState tolerates a legacy ledger with a bare provider column', () => { + const dir = tmp(); + const file = path.join(dir, 'state_5.sqlite'); + const db = new DatabaseSync(file); + db.exec('CREATE TABLE threads (id TEXT PRIMARY KEY, thread_source TEXT, provider TEXT);'); + db.prepare('INSERT INTO threads VALUES (?,?,?)').run('t-1', 'user', 'azure'); + db.close(); + const ledger = readCodexState({ dir }); + assert.equal(ledger.threads.get('t-1').provider, 'azure'); +}); + test('readCodexState degrades to null when the load-bearing columns are missing', () => { const dir = tmp(); const file = path.join(dir, 'state_9.sqlite'); diff --git a/tests/kit/live-adapters.test.mjs b/tests/kit/live-adapters.test.mjs index 8de1329..d7464d5 100644 --- a/tests/kit/live-adapters.test.mjs +++ b/tests/kit/live-adapters.test.mjs @@ -57,6 +57,43 @@ test('Codex adapter handles session, tool call and tool result fixture shapes', assert.ok(!JSON.stringify(done).includes('private')); }); +test('Codex session_meta model_provider becomes an observed provider claim', () => { + const [started] = adaptCodexRecord({ + type: 'session_meta', timestamp: now, + payload: { id: 'x1', model: 'gpt-x', model_provider: 'openrouter' }, + }, { observedAt: now }); + assert.equal(started.provider, 'openrouter'); + assert.equal(started.actor.provider, 'openrouter'); + assert.equal(started.providerProvenance, 'observed'); +}); + +test('Claude provider resolved from configuration keeps configured provenance', () => { + const [event] = adaptClaudeRecord({ + type: 'assistant', sessionId: 'c1', timestamp: now, + message: { model: 'claude-x', content: [] }, + }, { observedAt: now, provider: 'bedrock', providerProvenance: 'configured' }); + assert.equal(event.provider, 'bedrock'); + assert.equal(event.providerProvenance, 'configured'); +}); + +test('Claude first-party default stays an inferred claim, never observed', () => { + const [event] = adaptClaudeRecord({ + type: 'assistant', sessionId: 'c1', timestamp: now, + message: { model: 'claude-x', content: [] }, + }, { observedAt: now, provider: 'anthropic', providerProvenance: 'inferred' }); + assert.equal(event.provider, 'anthropic'); + assert.equal(event.providerProvenance, 'inferred'); +}); + +test('Claude without any provider evidence reports unknown provenance', () => { + const [event] = adaptClaudeRecord({ + type: 'assistant', sessionId: 'c1', timestamp: now, + message: { model: 'claude-x', content: [] }, + }, { observedAt: now }); + assert.equal(event.provider, null); + assert.equal(event.providerProvenance, 'unknown'); +}); + test('Codex ledger creates authoritative spawn edges', () => { const ledger = { parents: new Map([['child', 'parent']]), diff --git a/tests/kit/live-claude-provider.test.mjs b/tests/kit/live-claude-provider.test.mjs new file mode 100644 index 0000000..41b3400 --- /dev/null +++ b/tests/kit/live-claude-provider.test.mjs @@ -0,0 +1,86 @@ +// claude-provider.mjs — who serves a Claude Code session. Selection mirrors the +// documented env surface (CLAUDE_CODE_USE_BEDROCK/_VERTEX/_FOUNDRY, +// ANTHROPIC_BASE_URL) across settings layers; provenance is configured/inferred. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import path from 'node:path'; +import { resolveClaudeProvider } from '../../src/lib/live/claude-provider.mjs'; + +const HOME = '/home/dev'; +const CWD = '/repo/app'; +const base = { cwd: CWD, env: {}, home: HOME, platform: 'linux', read: () => null }; +const files = (map) => (file) => map[file] ?? null; +const userSettings = path.join(HOME, '.claude', 'settings.json'); +const projectShared = path.join(CWD, '.claude', 'settings.json'); +const projectLocal = path.join(CWD, '.claude', 'settings.local.json'); +const managed = '/etc/claude-code/managed-settings.json'; + +test('defaults to anthropic with inferred provenance when nothing selects a provider', () => { + assert.deepEqual(resolveClaudeProvider(base), + { provider: 'anthropic', provenance: 'inferred' }); +}); + +test('shell CLAUDE_CODE_USE_BEDROCK selects bedrock as configured', () => { + assert.deepEqual(resolveClaudeProvider({ ...base, env: { CLAUDE_CODE_USE_BEDROCK: '1' } }), + { provider: 'bedrock', provenance: 'configured' }); +}); + +test('an explicit "0"/"false" flag is not a selection', () => { + assert.equal(resolveClaudeProvider({ + ...base, env: { CLAUDE_CODE_USE_BEDROCK: '0', CLAUDE_CODE_USE_VERTEX: 'false' }, + }).provider, 'anthropic'); +}); + +test('user settings env selects vertex without any shell variable', () => { + const read = files({ [userSettings]: { env: { CLAUDE_CODE_USE_VERTEX: '1' } } }); + assert.deepEqual(resolveClaudeProvider({ ...base, read }), + { provider: 'vertex', provenance: 'configured' }); +}); + +test('project settings outrank user settings; local outranks shared', () => { + const read = files({ + [userSettings]: { env: { CLAUDE_CODE_USE_VERTEX: '1' } }, + [projectShared]: { env: { CLAUDE_CODE_USE_VERTEX: '', CLAUDE_CODE_USE_BEDROCK: '1' } }, + [projectLocal]: { env: { CLAUDE_CODE_USE_BEDROCK: '', CLAUDE_CODE_USE_FOUNDRY: '1' } }, + }); + assert.deepEqual(resolveClaudeProvider({ ...base, read }), + { provider: 'foundry', provenance: 'configured' }); +}); + +test('managed settings outrank everything', () => { + const read = files({ + [managed]: { env: { CLAUDE_CODE_USE_BEDROCK: '' } }, + [projectLocal]: { env: { CLAUDE_CODE_USE_BEDROCK: '1' } }, + }); + assert.equal(resolveClaudeProvider({ ...base, read }).provider, 'anthropic'); +}); + +test('an empty settings value masks a shell variable (documented unset idiom)', () => { + const read = files({ [userSettings]: { env: { ANTHROPIC_BASE_URL: '' } } }); + assert.deepEqual(resolveClaudeProvider({ + ...base, read, env: { ANTHROPIC_BASE_URL: 'https://proxy.corp.example' }, + }), { provider: 'anthropic', provenance: 'inferred' }); +}); + +test('ANTHROPIC_BASE_URL classifies openrouter, anthropic, and unknown gateways', () => { + const at = (url) => resolveClaudeProvider({ ...base, env: { ANTHROPIC_BASE_URL: url } }); + assert.deepEqual(at('https://openrouter.ai/api'), + { provider: 'openrouter', provenance: 'configured' }); + assert.deepEqual(at('https://api.anthropic.com'), + { provider: 'anthropic', provenance: 'configured' }); + assert.deepEqual(at('https://litellm.corp.example/v1'), + { provider: 'gateway', provenance: 'configured' }); + assert.deepEqual(at('not a url'), + { provider: 'gateway', provenance: 'configured' }); +}); + +test('without a cwd only user/managed layers are consulted', () => { + const read = files({ [projectLocal]: { env: { CLAUDE_CODE_USE_BEDROCK: '1' } } }); + assert.equal(resolveClaudeProvider({ ...base, cwd: undefined, read }).provider, 'anthropic'); +}); + +test('a malformed settings env block never breaks resolution', () => { + assert.equal(resolveClaudeProvider({ + ...base, read: () => ({ env: ['not', 'a', 'plain', 'object'] }), + }).provider, 'anthropic'); +}); diff --git a/tests/kit/live-core.test.mjs b/tests/kit/live-core.test.mjs index 6fb371c..d3f38b6 100644 --- a/tests/kit/live-core.test.mjs +++ b/tests/kit/live-core.test.mjs @@ -149,6 +149,26 @@ test('projection preserves safe display metadata and event provenance on resourc assert.equal(node.durationMs, 125); }); +test('projection retains provider, provenance and model when later evidence lacks them', () => { + const identified = { + ...createLiveEvent(base({ + actor: { id: 's1', kind: 'session', provider: 'openai', model: 'gpt-x' }, + source: { adapter: 'fixture', confidence: 'observed', fields: { provider: 'observed' } }, + })), eventId: 'ak:1', + }; + const anonymous = { + ...createLiveEvent(base({ + actor: { id: 's1', kind: 'session' }, action: 'agent.output', + })), eventId: 'ak:2', + }; + const projection = reduceLiveEvent(reduceLiveEvent(emptyLiveProjection(), identified), anonymous); + const node = projection.sessions.get('claude:s1').nodes.get('s1'); + assert.equal(node.provider, 'openai'); + assert.equal(node.providerProvenance, 'observed'); + assert.equal(node.model, 'gpt-x'); + assert.equal(node.lastAction, 'agent.output'); +}); + test('projection is idempotent by eventId and preserves terminal state', () => { const started = { ...createLiveEvent(base()), eventId: 'ak:1', ingestSeq: 1, diff --git a/tests/kit/live-service.test.mjs b/tests/kit/live-service.test.mjs index 585c513..c23c8cc 100644 --- a/tests/kit/live-service.test.mjs +++ b/tests/kit/live-service.test.mjs @@ -56,6 +56,59 @@ test('service bootstraps safe metadata then tails existing files from end', asyn assert.equal(service.snapshot().health.claude.status, 'ok'); }); +test('service resolves the Claude inference provider from configuration, keyed by raw cwd', (t) => { + const sb = sandbox(); + fs.writeFileSync(path.join(sb.claude, 'c1.jsonl'), line({ + type: 'user', sessionId: 'c1', timestamp: '2026-07-27T10:00:00Z', + cwd: '/Users/private-user/work/visible-project', + message: { model: 'claude-x', content: 'private' }, + })); + const asked = []; + const service = new LiveSessionsService({ + roots: sb.roots, intervalMs: 10, readCodexState: () => null, + resolveClaudeProvider: ({ cwd }) => { + asked.push(cwd); + return { provider: 'vertex', provenance: 'configured' }; + }, + now: () => '2026-07-27T12:00:00Z', + }); + t.after(() => service.close()); + service.start(); + assert.deepEqual(asked, ['/Users/private-user/work/visible-project']); + const node = service.snapshot().sessions[0].nodes.find((n) => n.kind === 'session'); + assert.equal(node.provider, 'vertex'); + assert.equal(node.providerProvenance, 'configured'); +}); + +test('codex metadata learned during bootstrap persists into live tailing', async (t) => { + const sb = sandbox(); + const file = path.join(sb.codex, 'rollout-2026-07-27T10-00-00-x1.jsonl'); + fs.writeFileSync(file, + line({ type: 'session_meta', timestamp: '2026-07-27T10:00:00Z', + payload: { id: 'x1', model: 'gpt-x', model_provider: 'openai', cwd: '/Users/private/repo' } }) + + line({ type: 'turn_context', timestamp: '2026-07-27T10:00:01Z', + payload: { model: 'gpt-x', cwd: '/Users/private/repo' } })); + const service = new LiveSessionsService({ + roots: sb.roots, intervalMs: 10, readCodexState: () => null, + now: () => '2026-07-27T12:00:00Z', + }); + t.after(() => service.close()); + const live = []; + service.subscribe((event) => live.push(event)); + service.start(); + fs.appendFileSync(file, line({ type: 'event_msg', timestamp: '2026-07-27T12:00:01Z', + payload: { type: 'agent_message', message: 'private' } })); + await waitUntil(() => live.some((event) => event.action === 'agent.output'), + 'expected the appended agent_message to surface'); + const output = live.find((event) => event.action === 'agent.output'); + assert.equal(output.sessionId, 'x1'); + assert.equal(output.provider, 'openai'); + assert.equal(output.providerProvenance, 'observed'); + const node = service.snapshot().sessions[0].nodes.find((n) => n.kind === 'session'); + assert.equal(node.provider, 'openai'); + assert.equal(node.providerProvenance, 'observed'); +}); + test('service discovers nested subagent transcripts and files them under the parent session', async (t) => { const sb = sandbox(); const nested = path.join(sb.claude, 'c1', 'subagents');