Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
owner, immutable request snapshots remain enforced at AgentRun acceptance and backend dispatch,
and SessionEvent-to-RuntimeEvent conversion remains a pure mapper.
- Retired the Task Ledger domain: SessionTodo is now the sole authority for in-session work items, and the operational-state schema drops the `workflow_task_ledger_events` table on first open. **Unfinished Tasks are not migrated and are permanently deleted.** This affects workspaces last opened by `v0.1.0` through `v0.1.11`, `cli-v0.1.0-beta.1`, `v0.2.0-incubating-rc1`, or a `v0.2.0-dev` build; those releases wrote Tasks to a table that no shipped build ever bridged into SessionTodo. Before opening such a workspace with this build, finish or export the Tasks you still need, or copy the workspace's `runtime.sqlite` aside — the migration removes the only live copy, so afterwards recovery requires a backup made in advance.
- Let the provider decide whether a request fits, and anchored the estimate that decides when to compact on the last request the provider actually counted. `token_usage` records now persist that anchor under a new `lastRequestAnchor` key. **Sessions this build writes do not open in earlier releases:** those decode `token_usage` against a closed allowlist, so the unknown key fails the record and, with it, the Session that contains it. Downgrading therefore needs a copy of the workspace's `runtime.sqlite` taken before the upgrade. Retired with the local verdict: nothing produces the `context_budget_exhausted` stop reason any more — a request that really is too large is compacted and retried once, then reported as a `context_overflow` provider error — though sessions that already recorded it still decode and present. The Runtime Host compatibility epoch moves to 94.
- Let the provider decide whether a request fits. Proactive compaction now uses only a user-declared Maka window and the previous accepted request's provider-reported `inputTokens + outputTokens`; no declaration means no proactive capacity threshold. `/models` and generated model metadata are display hints, not limits. `token_usage` records persist the last-request anchor under `lastRequestAnchor`; its new `{ inputTokens, outputTokens }` shape still decodes the retired `payloadChars` key from older sessions. Requests that are too large are compacted and retried once after a real provider rejection, then reported as a `context_overflow` provider error. New provider-dropping, context-window suggestion, context-window overrun and reported-window-exceeded system notes explain provider-side context changes, an exchange that ran past the declared window, and, once per crossing, a provider that accepts a request past the window its own model reports while nothing is declared. The reply reserve that arms the proactive threshold is twice the last real reply, bounded at 8,000 tokens, rather than the model's maximum output. **Sessions this build writes do not open in earlier releases:** those decode `token_usage` against a closed allowlist, so the reshaped `lastRequestAnchor` key fails the record and, with it, the Session that contains it; downgrading therefore needs a copy of the workspace's `runtime.sqlite` taken before the upgrade. Nothing produces the `context_budget_exhausted` stop reason any more — a request that really is too large is compacted and retried once, then reported as a `context_overflow` provider error — though sessions that already recorded it still decode and present. The Runtime Host compatibility epoch moves to 105.
- Unified context management under one Runtime-owned policy. `MAKA_CONTEXT_*` environment overrides no longer tune or disable compaction and Tool Result pruning; model-visible archive placeholders are read on demand through bounded `ArchiveRead` calls instead of eager hydration. Previously supported overrides are ignored on upgrade: if Tool Result pruning was set to `off`, pruning is re-enabled, and there is currently no supported replacement opt-out.
- Moved Read image snapshots into the durable context-offload store with Runtime-owned
lifecycle identity, exact branch and revision copying, recovery-safe cleanup, and bounded
Expand Down
76 changes: 38 additions & 38 deletions docs/architecture/llm-compaction-events-log-projection-draft.md

Large diffs are not rendered by default.

Large diffs are not rendered by default.

39 changes: 39 additions & 0 deletions packages/cli/src/pi-transcript.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1345,6 +1345,45 @@ function systemNoteText(message: SystemNoteMessage): string | undefined {
return 'Context compacted to keep this task within the model window.';
case 'context_compaction_failed_open':
return 'Context summary failed; the session continued without a new summary.';
case 'context_provider_dropping':
return 'The provider is dropping or rewriting context: content was appended but its reported usage did not grow. Declare a context window for this model so Maka compacts first.';
case 'context_reported_window_exceeded': {
const data = message.data as
| { usedTokens?: unknown; reportedContextWindow?: unknown }
| undefined;
const used = typeof data?.usedTokens === 'number' ? data.usedTokens : undefined;
const reported =
typeof data?.reportedContextWindow === 'number' ? data.reportedContextWindow : undefined;
if (used === undefined || reported === undefined) {
return 'This exchange ran past the context window this model reports, and the provider accepted it anyway.';
}
return `This exchange used about ${used} tokens, past the ${reported} this model reports, and the provider accepted it without complaint. Nothing is declared, so Maka does not compact on its own; declare a context window to have it compact first.`;
}
case 'context_window_overrun': {
const data = message.data as
| { usedTokens?: unknown; declaredContextWindow?: unknown }
| undefined;
const used = typeof data?.usedTokens === 'number' ? data.usedTokens : undefined;
const declared =
typeof data?.declaredContextWindow === 'number' ? data.declaredContextWindow : undefined;
if (used === undefined || declared === undefined) {
return 'This exchange ran past the context window declared for this model.';
}
return `This exchange used about ${used} tokens against the declared window of ${declared}: the reply needed more room than was left. Maka compacts before the next request; raise the window if the replies should stay whole.`;
}
case 'context_window_suggestion': {
const data = message.data as
| { suggestedContextWindow?: unknown; declaredContextWindow?: unknown }
| undefined;
const tokens =
typeof data?.suggestedContextWindow === 'number' ? data.suggestedContextWindow : undefined;
const declared =
typeof data?.declaredContextWindow === 'number' ? data.declaredContextWindow : undefined;
if (tokens === undefined) return 'The provider rejected this request as too large.';
return declared === undefined
? `The provider rejected this request. No context window is declared for this model; the last accepted request was about ${tokens} tokens — declare that as the window so Maka compacts first.`
: `The provider rejected this request at about ${tokens} tokens, below the declared window of ${declared}. The declaration is likely larger than the provider's window; consider lowering it to ${tokens}.`;
}
case 'step_limit':
return STEP_LIMIT_NOTICE_TEXT;
case 'error':
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,28 +24,33 @@ import { decodeCanonicalMessage } from '../session.js';

const usage = { input: 370, output: 60 };

test('a last-request anchor is only valid as a complete positive pair', () => {
test('a last-request anchor accepts the new usage shape and retired payload key', () => {
assert.equal(isLastRequestAnchor({ inputTokens: 120, outputTokens: 30 }), true);
assert.equal(isLastRequestAnchor({ inputTokens: 120 }), true);
assert.equal(isLastRequestAnchor({ inputTokens: 120, payloadChars: 4_000 }), true);
assert.equal(isLastRequestAnchor({ inputTokens: 120 }), false);
assert.equal(isLastRequestAnchor({ payloadChars: 4_000 }), false);
assert.equal(isLastRequestAnchor({ inputTokens: 0, payloadChars: 4_000 }), false);
assert.equal(isLastRequestAnchor({ inputTokens: 120, payloadChars: 0 }), false);
assert.equal(
isLastRequestAnchor({ inputTokens: 120, payloadChars: 4_000, stepNumber: 2 }),
false,
);
assert.equal(isLastRequestAnchor({ inputTokens: 120, outputTokens: -1 }), false);
assert.equal(isLastRequestAnchor({ inputTokens: 120, foo: 1 }), false);
});

test('token-usage fields carry the anchor and reject a broken one', () => {
assert.equal(
isTokenUsageFields({ ...usage, lastRequestAnchor: { inputTokens: 120, payloadChars: 4_000 } }),
isTokenUsageFields({ ...usage, lastRequestAnchor: { inputTokens: 120, outputTokens: 30 } }),
true,
);
assert.equal(isTokenUsageFields(usage), true);
assert.equal(isTokenUsageFields({ ...usage, lastRequestAnchor: { inputTokens: 120 } }), false);
assert.equal(
isTokenUsageFields({ ...usage, lastRequestAnchor: { inputTokens: 120, payloadChars: 4_000 } }),
true,
);
assert.equal(
isTokenUsageFields({ ...usage, lastRequestAnchor: { inputTokens: 120, foo: 1 } }),
false,
);
});

test('a half-written anchor fails the whole token_usage message decode', () => {
test('an invalid anchor fails the whole token_usage message decode', () => {
const message = {
type: 'token_usage',
id: 'usage-1',
Expand All @@ -56,11 +61,11 @@ test('a half-written anchor fails the whole token_usage message decode', () => {
assert.deepEqual(
decodeCanonicalMessage({
...message,
lastRequestAnchor: { inputTokens: 120, payloadChars: 4_000 },
lastRequestAnchor: { inputTokens: 120, outputTokens: 30 },
}),
{ ...message, lastRequestAnchor: { inputTokens: 120, payloadChars: 4_000 } },
{ ...message, lastRequestAnchor: { inputTokens: 120, outputTokens: 30 } },
);
assert.throws(() =>
decodeCanonicalMessage({ ...message, lastRequestAnchor: { payloadChars: 4_000 } }),
decodeCanonicalMessage({ ...message, lastRequestAnchor: { inputTokens: 0 } }),
);
});
33 changes: 33 additions & 0 deletions packages/core/src/model-thinking.ts
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,39 @@ export function relayModelProfile(
return normalizeRelayModelProfile(connection.relayModelProfiles?.[modelId]);
}

/** The connection fields the declared-window rule reads; structural so runtime and UI projections both fit. */
export interface DeclaredContextWindowContext extends ConnectionThinkingContext {
readonly models?: readonly {
readonly id: string;
readonly contextWindow?: number;
readonly inputLimit?: number;
readonly factOverriddenFields?: readonly string[];
}[];
}

/**
* The context window the USER declared for a model — the Maka window: the
* proactive compaction target, and nothing else. Exactly two sources count as
* a declaration: a model-facts pin (`factOverriddenFields` includes
* `contextWindow`, narrowest of window/input limit) and a relay model profile.
* A provider's `/models` report and generated metadata describe the model and
* are shown as a hint; they never become a threshold on their own. This is the
* single owner of that rule for runtime and UI (#4559).
*/
export function declaredContextWindow(
connection: DeclaredContextWindowContext,
modelId: string,
): number | undefined {
const model = connection.models?.find((candidate) => candidate.id === modelId);
if (model?.factOverriddenFields?.includes('contextWindow')) {
const values = [model.contextWindow, model.inputLimit].filter(
(value): value is number => typeof value === 'number' && Number.isFinite(value) && value > 0,
);
return values.length > 0 ? Math.min(...values) : undefined;
}
return relayModelProfile(connection, modelId)?.contextWindow;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Real edge case from live data: kimi-coding-plan/k3-256k accepts requests beyond its fetched window without rejecting or truncating — observed a session accepted at 322K input tokens against a fetched contextWindow of 262,144, with usage still growing monotonically (305K → 322K). On such a provider every signal this design reads stays dark for an undeclared window:

  • no rejection → reactive fold never fires;
  • usage never plateaus → context_provider_dropping never fires (and it is mid-turn only anyway);
  • context_window_suggestion only fires on a surfaced rejection → never written;
  • a fetched /models window is not a declaration → the proactive trigger below never arms.

Net effect: an undeclared kimi session degrades silently and indefinitely (this is exactly the data behind #4634). I agree with "Maka does not decide for the user" as a principle, but for providers observed to accept over-window requests, could the fetched window seed a default declaration (user-overridable), or should the cross-turn plateau check (#4623 PR D) be a blocker rather than a follow-up?

}

/**
* Mirrors @ai-sdk/openai@4.0.42 priority-processing detection. The UI and
* runtime share this gate so a saved Fast declaration always reaches the wire.
Expand Down
12 changes: 12 additions & 0 deletions packages/core/src/session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -792,6 +792,10 @@ export function userFacingText(message: Pick<UserMessage, 'text' | 'displayText'
const USER_VISIBLE_SESSION_SYSTEM_NOTES = new Set([
'context_compacted',
'context_compaction_failed_open',
'context_provider_dropping',
'context_window_suggestion',
'context_window_overrun',
'context_reported_window_exceeded',
'step_limit',
]);

Expand Down Expand Up @@ -1155,6 +1159,10 @@ export interface SystemNoteMessage {
| 'model_change'
| 'context_compacted'
| 'context_compaction_failed_open'
| 'context_provider_dropping'
| 'context_window_suggestion'
| 'context_window_overrun'
| 'context_reported_window_exceeded'
| 'step_limit'
| 'error'
| 'abort';
Expand Down Expand Up @@ -1399,6 +1407,10 @@ const SYSTEM_NOTE_KINDS = new Set([
'model_change',
'context_compacted',
'context_compaction_failed_open',
'context_provider_dropping',
'context_window_suggestion',
'context_window_overrun',
'context_reported_window_exceeded',
'step_limit',
'error',
'abort',
Expand Down
45 changes: 28 additions & 17 deletions packages/core/src/usage-record-schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,6 @@ const CURRENT_CONTEXT_BUDGET_SHAPE = defineObjectShape<ContextBudgetDiagnostic>(
],
[
'policyName',
'maxHistoryEstimatedTokens',
'prunedToolResults',
'prunedToolResultEstimatedTokensBefore',
'prunedToolResultEstimatedTokensAfter',
Expand All @@ -105,6 +104,7 @@ const CURRENT_CONTEXT_BUDGET_SHAPE = defineObjectShape<ContextBudgetDiagnostic>(
* produce them through ContextBudgetDiagnostic.
*/
const RETIRED_CONTEXT_BUDGET_KEYS = [
'maxHistoryEstimatedTokens',
'maxHistoryTurns',
'semanticCompactEnabled',
'semanticCompactMode',
Expand Down Expand Up @@ -285,38 +285,49 @@ export interface TokenUsageFields {
providerRequestTraceId?: string;
/**
* The send's LAST provider request, as a pair: the input tokens the provider
* reported for it, and the wire payload chars the runtime measured for that
* same request. `input` above is the send's sum across steps and cannot
* anchor anything; this pair can, so the next turn estimates its first
* request from real usage instead of guessing the whole payload at char/4.
*
* Only the pair means anything — an anchor taken from one request and a
* baseline from another is off by a whole step's growth — so the two numbers
* live in one object that is written and read together. Absent means no
* anchor, and the estimate falls back to the cold start.
* reported for it, and its output tokens. `input` above is the send's sum
* across steps and cannot anchor anything; the last step's real input and
* output can, so the next turn judges its first request from real usage.
* Absent means no anchor, and the next turn has no proactive fold until its
* first accepted request.
*/
lastRequestAnchor?: LastRequestAnchor;
}

/** Real input tokens of one provider request, paired with its measured payload chars. */
/**
* The last provider request of a send, as the provider counted it: its real
* input tokens and its real output tokens. Together they are the baseline the
* next request is judged from — everything the model produced is re-sent as
* input — with no local measure involved (#4559).
*
* `payloadChars` is a retired key from the 0.2.0 anchor, which paired the input
* count with a locally measured payload size. It is still accepted on decode so
* sessions written by that build keep loading, and ignored.
*/
export interface LastRequestAnchor {
inputTokens: number;
payloadChars: number;
outputTokens?: number;
}

const LAST_REQUEST_ANCHOR_SHAPE = defineObjectShape<LastRequestAnchor>()(
['inputTokens', 'payloadChars'],
[],
['inputTokens'],
['outputTokens'],
);
const RETIRED_LAST_REQUEST_ANCHOR_KEYS = ['payloadChars'] as const;
const LAST_REQUEST_ANCHOR_DECODE_SHAPE = {
required: LAST_REQUEST_ANCHOR_SHAPE.required,
allowed: new Set([...LAST_REQUEST_ANCHOR_SHAPE.allowed, ...RETIRED_LAST_REQUEST_ANCHOR_KEYS]),
};

export function isLastRequestAnchor(value: unknown): value is LastRequestAnchor {
return (
isRecord(value) &&
hasExactShape(value, LAST_REQUEST_ANCHOR_SHAPE) &&
hasExactShape(value, LAST_REQUEST_ANCHOR_DECODE_SHAPE) &&
isFiniteNumber(value.inputTokens) &&
isFiniteNumber(value.payloadChars) &&
value.inputTokens > 0 &&
value.payloadChars > 0
(value.outputTokens === undefined ||
(isFiniteNumber(value.outputTokens) && value.outputTokens >= 0)) &&
(value.payloadChars === undefined || isFiniteNumber(value.payloadChars))
);
}

Expand Down
1 change: 0 additions & 1 deletion packages/core/src/usage-stats/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,6 @@ export interface CompactionDecisionDiagnostic {
export interface ContextBudgetDiagnostic {
enabled: boolean;
policyName?: string;
maxHistoryEstimatedTokens?: number;
estimatedTokensBefore: number;
estimatedTokensAfter: number;
keptTurns: number;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1779,6 +1779,16 @@ test('production Host executes a canonical ai-sdk Session against a real provide
});
assert.equal(configured.kind, 'committed');
await publishConnectionModel(policy, connection.connectionId, MODEL_ID);
// The fetched /models value is metadata only. This explicit model-facts
// declaration is the Maka compaction target used by the long-session flow.
await writeFile(
join(root, 'model-facts.json'),
JSON.stringify({
schemaVersion: 1,
overrides: { [`moonshot:${MODEL_ID}`]: { contextWindow: 3_072 } },
}),
'utf8',
);
let policySnapshot = await policy.runtimePolicy.getSnapshot();
const personalized = await policy.runtimePolicy.mutate({
expectedRevision: policySnapshot.revision,
Expand Down Expand Up @@ -1860,8 +1870,8 @@ test('production Host executes a canonical ai-sdk Session against a real provide
assert.equal(remembered.result.kind, 'committed');

const turnIds: string[] = [];
// Cross the history high-water without making the text-only compact input
// exceed this fixture's 2,304-token summarizer budget.
// Cross the explicitly declared Maka window without making the text-only
// compact input exceed this fixture's 2,304-token summarizer budget.
for (let index = 0; index < 5; index += 1) {
const turnId = randomUUID();
turnIds.push(turnId);
Expand Down
10 changes: 9 additions & 1 deletion packages/runtime-host/src/protocol/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,15 @@ export const RUNTIME_HOST_REGISTRATION_SCHEMA_VERSION = 1 as const;
export const RUNTIME_HOST_PROTOCOL_VERSION = 0 as const;
// Increment when the same protocol version no longer guarantees safe Client-Host
// interoperability. Mismatches are rejected before domain commands are admitted.
export const RUNTIME_HOST_COMPATIBILITY_EPOCH = 104 as const;
export const RUNTIME_HOST_COMPATIBILITY_EPOCH = 105 as const;
// 105: Session transcripts gain four `system_note` kinds
// (`context_provider_dropping`, `context_window_suggestion`,
// `context_window_overrun`, `context_reported_window_exceeded`) and
// `token_usage`
// records reshape `lastRequestAnchor` to `{ inputTokens, outputTokens }`, all
// behind closed allowlists in @maka/core. An older client that handshakes
// would fail `decodeStoredMessage` on the first transcript carrying them, so
// the pair must refuse each other at the handshake instead (#4559).
// 104: WorkHub Coordination actions add closed direct-stop proposals,
// confirmations, expected-state preconditions, and outcomes. Older peers
// reject these strict shapes.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -392,7 +392,7 @@ async function buildHostAiSdkBackend(
: {}),
...(!input.context.tools && input.childAgents ? input.childAgents : {}),
providerOptions,
contextBudget: buildDefaultContextBudgetPolicy(target.connection, {
contextBudget: buildDefaultContextBudgetPolicy({
name: 'runtime-host-default-history-budget',
modelId: target.model,
}),
Expand Down
Loading