Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion docs/LIVE-SESSIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,11 @@ request. Stopping the dashboard closes the live service and all clients.
stores and bootstrapped from bounded, metadata-only records.
- Project-first session cards with a host glyph and name, independently
evidenced inference provider/model when reported, lifecycle, freshness, and
a concise operational summary.
a concise operational summary. Provider identity is graded (ADR-0021):
observed from Codex artifacts (`model_provider`), configured/inferred from
Claude Code's documented selection surface (Bedrock, Vertex, Foundry,
`ANTHROPIC_BASE_URL` gateways such as OpenRouter), and "Provider not
established" only when no evidence of any grade exists.
- Session, agent, sub-agent, tool, skill, plugin, MCP, and gate entities when a
supported source record identifies them.
- Authoritative Codex parent/child edges from the Codex state ledger.
Expand Down
67 changes: 67 additions & 0 deletions docs/adr/0021-inference-provider-provenance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# ADR-0021 — Inference-provider provenance for live sessions

- **Status:** Accepted
- **Date:** 2026-07-31
- **Deciders:** agentic-kit maintainers
- **Related:** [ADR-0012](0012-live-sessions-observability.md)

## Context

ADR-0012 keeps the execution-host and inference-provider axes independent and forbids promoting
a host-based assumption to an observed provider claim. In practice every Claude Code session in
the Live view rendered "Provider not established", and — unnoticed — every Codex session did too.
Investigation against current upstream sources found two distinct causes:

1. **Codex records the provider; ak read the wrong field.** Codex rollout `session_meta`
payloads and the `threads` table of `state_N.sqlite` both spell the field `model_provider`
(verified live against `~/.codex` and openai/codex session examples). The adapters and the
state reader looked for a bare `provider` column/key that does not exist, so real observed
evidence — including custom `model_providers` entries such as `openrouter`, `azure`, or
`ollama` from `~/.codex/config.toml` — was silently dropped by the schema-tolerant column
filter.
2. **Claude Code genuinely never writes the provider.** No record type in
`~/.claude/projects/*/*.jsonl` carries one. Which endpoint serves a session is decided by the
host's documented configuration surface instead: `CLAUDE_CODE_USE_BEDROCK`,
`CLAUDE_CODE_USE_VERTEX`, `CLAUDE_CODE_USE_FOUNDRY`, and `ANTHROPIC_BASE_URL` (gateway/proxy),
where settings-file `env` blocks override the shell with managed > project local > project
shared > user precedence and an empty string means "unset at this layer"
(code.claude.com/docs/en/env-vars and the Bedrock/Vertex/Foundry deployment guides).

The event schema already reserved the vocabulary for this: `providerProvenance ∈ observed |
configured | inferred | unknown`, populated from field-level evidence. Only emitters were missing.

## Decision

- **Codex: read `model_provider`** in the rollout adapter and the state-ledger reader, keeping
the bare `provider` spelling only as legacy tolerance. The claim remains **observed** — it is
in the artifact. Custom provider ids pass through untranslated.
- **Claude: resolve the provider from the host's configuration surface** at transcript-discovery
time (`src/lib/live/claude-provider.mjs`), mirroring the documented selection order:
Bedrock flag → Vertex flag → Foundry flag → `ANTHROPIC_BASE_URL` (classified to `anthropic`,
`openrouter`, or generic `gateway` by hostname) → first-party default `anthropic`. Explicit
selections are **configured**; the first-party default is **inferred**; an unknown gateway stays
a `gateway`, never a guessed vendor. Resolution is memoized per session `cwd` and injectable
for tests.
- **Provenance is carried, never upgraded.** Adapters put the resolution's own provenance into
`source.fields.provider`; the schema lifts it into `providerProvenance`. Nothing configured or
inferred may be displayed as observed, per ADR-0012.
- **Identity evidence is retained.** Projection (server and browser reducers alike) keeps an
established provider/model claim when a later event for the same node carries none — absence of
evidence on one event does not erase evidence already observed. Metadata learned while
bootstrapping a transcript (codex `session_meta` identity in particular) persists into live
tailing for the same reason.
- **Presentation** names the new values (AWS Bedrock, Google Vertex AI, Microsoft Foundry,
Custom gateway) and continues to render "Provider not established" only when no evidence of any
grade exists.

## Consequences

- Codex sessions now show their serving provider with observed provenance, including
OpenRouter/Azure/Ollama-served sessions configured through `model_providers`.
- Claude Code sessions show who serves Claude models — Anthropic, Bedrock, Vertex, Foundry,
OpenRouter, or an unnamed gateway — labeled configured or inferred, answering the operator
question without overclaiming.
- OpenCode is unaffected: it is not a live host (ADR-0017/0018 scope it to managed execution),
and its usage-side records already carry an observed `providerID`.
- If codex renames or Claude Code starts recording the serving endpoint, the adapters prefer the
in-artifact (observed) value automatically.
7 changes: 7 additions & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ Consequences**, and cites the grounded source it rests on where relevant.
| [0018](0018-generalized-host-worker-execution.md) | Generalized host-worker execution; `ak run` canonical | Accepted; compatibility amended |
| [0019](0019-escalation-in-ak-run.md) | Bounded per-worker escalation in `ak run` | Accepted; historical context closed |
| [0020](0020-ga-stable-surfaces.md) | One stable GA surface per capability | Implemented |
| [0021](0021-inference-provider-provenance.md) | Inference-provider provenance for live sessions | Accepted |

Theme: ADRs **0001–0006** define **dual-host LLM routing and leadership** — how `ak` lets ruflo route
each development activity (architecture, implementation, testing, review, …) to the right host (Claude
Expand Down Expand Up @@ -99,3 +100,9 @@ clauses are superseded by ADR-0020.
host-management namespace, and the versioned top-level routing envelope; removes dynamic
adapter-specific execution bootstrap; limits old vocabulary to marked decision history and one
upgrade section; and preserves OpenCode's opt-in, supervised, non-primary, non-AQE boundary.

**0021** makes the Live view answer who serves each session's models without overclaiming. Codex's
in-artifact `model_provider` (rollouts + state ledger) is read as observed evidence; Claude Code's
provider is resolved from its documented configuration surface (Bedrock/Vertex/Foundry flags,
`ANTHROPIC_BASE_URL` gateways) with configured/inferred provenance, since its transcripts never
record the serving endpoint.
8 changes: 8 additions & 0 deletions docs/ddd/live-sessions.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,14 @@ multiple relationship types.
Project, provider, model, role, lifecycle, and hierarchy each retain their own provenance. A host
may suggest a provider, but that assumption cannot be displayed as an observed provider fact.

Provider evidence is graded by where it came from (ADR-0021). Codex artifacts name their serving
provider (`model_provider` in rollout `session_meta` and the state ledger), so Codex claims are
**observed**. Claude transcripts never name one; the domain resolves it from the host's documented
configuration surface — Bedrock/Vertex/Foundry selection flags and `ANTHROPIC_BASE_URL` gateway
classification across settings layers — yielding a **configured** claim, or an **inferred** one for
the first-party default. An unrecognized gateway stays `gateway` rather than a guessed vendor, and
a claim's grade is never upgraded downstream.

### `Activity`

An activity represents a bounded invocation or evaluation. It has source-scoped identity, actor,
Expand Down
7 changes: 5 additions & 2 deletions src/lib/codex-state.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ export function readCodexState(opts = {}) {
const pick = ['id', 'thread_source']
.concat([
'tokens_used', 'source', 'model', 'git_branch', 'agent_nickname', 'agent_role',
'cwd', 'provider', 'status',
'cwd', 'model_provider', 'provider', 'status',
].filter((c) => cols.has(c)));
const threads = new Map();
for (const row of db.prepare(`SELECT ${pick.join(', ')} FROM threads`).all()) {
Expand All @@ -75,7 +75,10 @@ export function readCodexState(opts = {}) {
agentNickname: typeof row.agent_nickname === 'string' ? row.agent_nickname : null,
agentRole: typeof row.agent_role === 'string' ? row.agent_role : null,
project: typeof row.cwd === 'string' ? resolveProjectLabel(row.cwd) : null,
provider: typeof row.provider === 'string' ? row.provider : null,
// codex's threads schema names this model_provider; the bare provider
// spelling is kept for tolerance of older/forked ledgers.
provider: typeof row.model_provider === 'string' ? row.model_provider
: (typeof row.provider === 'string' ? row.provider : null),
status: typeof row.status === 'string' ? row.status : null,
});
}
Expand Down
8 changes: 4 additions & 4 deletions src/lib/dashboard/live/client.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ export const LIVE_JS = `
function hostOf(v){return String(v&&v.host||"internal").toLowerCase();}
function hostName(v){var h=hostOf(v);return{claude:"Claude Code",codex:"Codex",internal:"Internal"}[h]||title(h);}
function hostGlyph(v){return{claude:"✳",codex:"◉",internal:"⌁"}[hostOf(v)]||"⌁";}
function inferenceProviderOf(v){if(v&&v.provider)return String(v.provider);var nodes=v&&v.nodes||[],session=nodes.find(function(n){return n.kind==="session"&&n.provider;});return session&&session.provider||null;}
function inferenceProviderName(v){var p=inferenceProviderOf(v);if(!p)return"Provider not established";return{anthropic:"Anthropic",openai:"OpenAI",openrouter:"OpenRouter",ollama:"Ollama",google:"Google Gemini"}[String(p).toLowerCase()]||title(p);}
function providerEvidence(v){var p=v&&v.providerProvenance;if(!p&&v&&v.nodes){var n=v.nodes.find(function(x){return x.kind==="session"&&x.provider;});p=n&&n.providerProvenance;}return p||"unknown";}
function inferenceProviderOf(v){if(v&&v.provider)return String(v.provider);var nodes=v&&v.nodes||[],session=nodes.find(function(n){return n.kind==="session"&&n.provider;})||nodes.find(function(n){return n.kind==="subagent"&&n.provider;});return session&&session.provider||null;}
function inferenceProviderName(v){var p=inferenceProviderOf(v);if(!p)return"Provider not established";return{anthropic:"Anthropic",openai:"OpenAI",openrouter:"OpenRouter",ollama:"Ollama",google:"Google Gemini",bedrock:"AWS Bedrock",vertex:"Google Vertex AI",foundry:"Microsoft Foundry",gateway:"Custom gateway"}[String(p).toLowerCase()]||title(p);}
function providerEvidence(v){var p=v&&v.providerProvenance;if(!p&&v&&v.nodes){var n=v.nodes.find(function(x){return x.kind==="session"&&x.provider;})||v.nodes.find(function(x){return x.kind==="subagent"&&x.provider;});p=n&&n.providerProvenance;}return p||"unknown";}
function kindName(k){return{session:"Coordinator",agent:"Agent",subagent:"Worker",tool:"Tool",skill:"Skill",plugin:"Plugin",mcp:"MCP tool",gate:"Quality gate"}[k]||title(k||"agent");}
function statusText(v){return{running:"Working",queued:"Queued",quiescent:"Waiting",expired:"No recent activity",blocked:"Blocked",completed:"Completed",failed:"Failed",cancelled:"Cancelled",idle:"Waiting"}[v]||"Outcome unavailable";}
function label(n){return n&&(n.toolName||n.label||n.toolCategory||n.role&&title(n.role)||kindName(n.kind))||"Activity";}
Expand All @@ -52,7 +52,7 @@ export const LIVE_JS = `
if(!ev||!ev.sessionId||!ev.actor||!ev.actor.id)return;if(ev.eventId&&state.seen[ev.eventId])return;
var sessions=state.snapshot.sessions.slice(),evKey=String(ev.sessionKey||ev.host+":"+ev.sessionId),i=sessions.findIndex(function(s){return sessionKey(s)===evKey;}),s=i>=0?Object.assign({},sessions[i]):{id:ev.sessionId,key:evKey,project:ev.project,projectKey:ev.projectKey,host:ev.host,status:"unknown",nodes:[],edges:[]};if(ev.parentSessionId)s.parentSessionId=ev.parentSessionId;
s.nodes=(s.nodes||[]).slice();s.edges=(s.edges||[]).slice();var targetTerminal=!!ev.target&&/\\.(completed|failed|cancelled)$/.test(ev.action||""),priorActor=s.nodes.find(function(n){return n.id===ev.actor.id;});
function upsert(a,status){var at=s.nodes.findIndex(function(n){return n.id===a.id;}),old=at>=0?s.nodes[at]:{},provider=a.provider||old.provider||null,next=Object.assign({},old,{id:a.id,kind:a.kind||old.kind,role:a.role||old.role,provider:provider,providerProvenance:provider?(ev.providerProvenance||old.providerProvenance||"unknown"):"unknown",model:a.model||old.model,label:a.label||old.label,toolName:a.toolName||old.toolName,host:ev.host||old.host,surface:ev.surface||old.surface,status:status||old.status,observedAt:ev.observedAt,confidence:ev.source&&ev.source.confidence||old.confidence,lastAction:ev.action,durationMs:ev.attributes&&ev.attributes.durationMs||old.durationMs});if(TERMINAL[old.status]&&!TERMINAL[next.status])next.status=old.status;if(at>=0)s.nodes[at]=next;else s.nodes.push(next);}
function upsert(a,status){var at=s.nodes.findIndex(function(n){return n.id===a.id;}),old=at>=0?s.nodes[at]:{},provider=a.provider||old.provider||null,next=Object.assign({},old,{id:a.id,kind:a.kind||old.kind,role:a.role||old.role,provider:provider,providerProvenance:provider?(a.provider?(ev.providerProvenance||"unknown"):(old.providerProvenance||ev.providerProvenance||"unknown")):"unknown",model:a.model||old.model,label:a.label||old.label,toolName:a.toolName||old.toolName,host:ev.host||old.host,surface:ev.surface||old.surface,status:status||old.status,observedAt:ev.observedAt,confidence:ev.source&&ev.source.confidence||old.confidence,lastAction:ev.action,durationMs:ev.attributes&&ev.attributes.durationMs||old.durationMs});if(TERMINAL[old.status]&&!TERMINAL[next.status])next.status=old.status;if(at>=0)s.nodes[at]=next;else s.nodes.push(next);}
upsert(ev.actor,targetTerminal?priorActor&&priorActor.status||"running":ev.status);if(ev.target&&ev.target.id){upsert(Object.assign({},ev.target,{label:ev.attributes&&(ev.attributes.toolName||ev.attributes.toolCategory)||ev.target.label,toolName:ev.attributes&&ev.attributes.toolName}),targetTerminal?ev.status:"running");var eid=ev.actor.id+"|"+ev.action+"|"+ev.target.id,edge={id:eid,source:ev.actor.id,target:ev.target.id,action:ev.action,confidence:ev.source&&ev.source.confidence},ei=s.edges.findIndex(function(x){return x.id===eid;});if(ei>=0)s.edges[ei]=edge;else s.edges.push(edge);}
if(ev.actor.kind==="session"&&/^session\\./.test(ev.action||""))s.status=ev.status;if(ev.status==="running"&&(!ev.source||ev.source.adapter!=="codex-state"))s.lifecycle="active";else if(!s.lifecycle)s.lifecycle="historical";s.updatedAt=ev.observedAt;if(i>=0)sessions[i]=s;else sessions.push(s);
state.snapshot={schemaVersion:state.snapshot.schemaVersion||1,cursor:ev.eventId||state.snapshot.cursor,projects:state.snapshot.projects,sessions:sessions,health:state.snapshot.health};if(ev.eventId)state.seen[ev.eventId]=1;state.events.unshift(ev);state.events=state.events.slice(0,100);state.lastAt=Date.now();if(!state.selected)selectSession(sessionKey(s),false);if(state.playback.mode==="live")render();
Expand Down
6 changes: 5 additions & 1 deletion src/lib/live/claude-adapter.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,11 @@ export function adaptClaudeRecord(record, context = {}) {
confidence: isSidechain && !record.agentId ? 'inferred' : 'observed',
fields: {
project: context.project ? 'observed' : null,
provider: record.provider || context.provider ? 'observed' : null,
// A provider on the record itself is observed evidence; one resolved
// from the host's configuration surface carries that resolution's own
// provenance (configured/inferred) and must not be upgraded.
provider: record.provider ? 'observed'
: (context.provider ? context.providerProvenance ?? 'configured' : null),
model: record.message?.model || context.model ? 'observed' : null,
status: 'observed',
hierarchy: isSidechain ? (record.agentId ? 'observed' : 'inferred') : 'observed',
Expand Down
78 changes: 78 additions & 0 deletions src/lib/live/claude-provider.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
import os from 'node:os';
import path from 'node:path';
import { readJson } from '../settings.mjs';

// Which endpoint serves a Claude Code session. Claude transcripts never record
// this, so the resolver mirrors the host's documented selection surface
// (code.claude.com/docs/en/env-vars + the Bedrock/Vertex/Foundry guides):
// CLAUDE_CODE_USE_BEDROCK / _VERTEX / _FOUNDRY switch the serving platform and
// ANTHROPIC_BASE_URL points the first-party path at a proxy or gateway.
// Settings `env` blocks override the shell — managed > project local >
// project shared > user — and an empty string means "unset at this layer".
// The result is a claim about configuration, so provenance is 'configured'
// (explicit selection found) or 'inferred' (first-party default), never
// 'observed'.

/** @type {Record<string, string>} */
const MANAGED_SETTINGS = {
darwin: '/Library/Application Support/ClaudeCode/managed-settings.json',
linux: '/etc/claude-code/managed-settings.json',
win32: 'C:\\ProgramData\\ClaudeCode\\managed-settings.json',
};

const plainEnv = (value) => value && typeof value === 'object' && !Array.isArray(value)
? value : null;

// "1" is the documented spelling; tolerate other truthy strings but never
// treat an explicit "0"/"false" as a selection.
const flag = (value) => typeof value === 'string' && value !== ''
&& !['0', 'false'].includes(value.toLowerCase());

function gatewayProvider(baseUrl) {
let host = null;
try { host = new URL(baseUrl).hostname.toLowerCase(); } catch { /* unparseable → generic */ }
if (host === 'api.anthropic.com' || host?.endsWith('.anthropic.com')) return 'anthropic';
if (host?.includes('openrouter')) return 'openrouter';
return 'gateway';
}

/**
* Resolve the inference provider configured for Claude Code sessions rooted at
* `cwd`. Returns { provider, provenance } — provenance 'configured' when an
* explicit selection was found, 'inferred' for the first-party default.
*
* @param {{ cwd?: string, env?: Record<string, string | undefined>, home?: string,
* platform?: string, read?: (file: string) => any }} [opts] test seams
*/
export function resolveClaudeProvider({
cwd,
env = process.env,
home = os.homedir(),
platform = process.platform,
read = readJson,
} = {}) {
const layers = [];
const managed = MANAGED_SETTINGS[platform];
if (managed) layers.push(plainEnv(read(managed)?.env));
if (typeof cwd === 'string' && cwd) {
layers.push(plainEnv(read(path.join(cwd, '.claude', 'settings.local.json'))?.env));
layers.push(plainEnv(read(path.join(cwd, '.claude', 'settings.json'))?.env));
}
layers.push(plainEnv(read(path.join(home, '.claude', 'settings.json'))?.env));
const lookup = (name) => {
for (const layer of layers) {
if (layer && Object.hasOwn(layer, name)) {
const value = layer[name];
return typeof value === 'string' && value !== '' ? value : null;
}
}
const value = env[name];
return typeof value === 'string' && value !== '' ? value : null;
};
if (flag(lookup('CLAUDE_CODE_USE_BEDROCK'))) return { provider: 'bedrock', provenance: 'configured' };
if (flag(lookup('CLAUDE_CODE_USE_VERTEX'))) return { provider: 'vertex', provenance: 'configured' };
if (flag(lookup('CLAUDE_CODE_USE_FOUNDRY'))) return { provider: 'foundry', provenance: 'configured' };
const base = lookup('ANTHROPIC_BASE_URL');
if (base) return { provider: gatewayProvider(base), provenance: 'configured' };
return { provider: 'anthropic', provenance: 'inferred' };
}
Loading
Loading