diff --git a/CHANGELOG.md b/CHANGELOG.md index c28db3872f..b4397922a7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,7 +48,7 @@ owner, immutable request snapshots remain enforced at AgentRun acceptance and backend dispatch, and SessionEvent-to-RuntimeEvent conversion remains a pure mapper. - Retired the Task Ledger domain: SessionTodo is now the sole authority for in-session work items, and the operational-state schema drops the `workflow_task_ledger_events` table on first open. **Unfinished Tasks are not migrated and are permanently deleted.** This affects workspaces last opened by `v0.1.0` through `v0.1.11`, `cli-v0.1.0-beta.1`, `v0.2.0-incubating-rc1`, or a `v0.2.0-dev` build; those releases wrote Tasks to a table that no shipped build ever bridged into SessionTodo. Before opening such a workspace with this build, finish or export the Tasks you still need, or copy the workspace's `runtime.sqlite` aside — the migration removes the only live copy, so afterwards recovery requires a backup made in advance. -- Let the provider decide whether a request fits, and anchored the estimate that decides when to compact on the last request the provider actually counted. `token_usage` records now persist that anchor under a new `lastRequestAnchor` key. **Sessions this build writes do not open in earlier releases:** those decode `token_usage` against a closed allowlist, so the unknown key fails the record and, with it, the Session that contains it. Downgrading therefore needs a copy of the workspace's `runtime.sqlite` taken before the upgrade. Retired with the local verdict: nothing produces the `context_budget_exhausted` stop reason any more — a request that really is too large is compacted and retried once, then reported as a `context_overflow` provider error — though sessions that already recorded it still decode and present. The Runtime Host compatibility epoch moves to 94. +- Let the provider decide whether a request fits. Proactive compaction now uses only a user-declared Maka window and the previous accepted request's provider-reported `inputTokens + outputTokens`; no declaration means no proactive capacity threshold. `/models` and generated model metadata are display hints, not limits. `token_usage` records persist the last-request anchor under `lastRequestAnchor`; its new `{ inputTokens, outputTokens }` shape still decodes the retired `payloadChars` key from older sessions. Requests that are too large are compacted and retried once after a real provider rejection, then reported as a `context_overflow` provider error. New provider-dropping, context-window suggestion, context-window overrun and reported-window-exceeded system notes explain provider-side context changes, an exchange that ran past the declared window, and, once per crossing, a provider that accepts a request past the window its own model reports while nothing is declared. The reply reserve that arms the proactive threshold is twice the last real reply, bounded at 8,000 tokens, rather than the model's maximum output. **Sessions this build writes do not open in earlier releases:** those decode `token_usage` against a closed allowlist, so the reshaped `lastRequestAnchor` key fails the record and, with it, the Session that contains it; downgrading therefore needs a copy of the workspace's `runtime.sqlite` taken before the upgrade. Nothing produces the `context_budget_exhausted` stop reason any more — a request that really is too large is compacted and retried once, then reported as a `context_overflow` provider error — though sessions that already recorded it still decode and present. The Runtime Host compatibility epoch moves to 105. - Unified context management under one Runtime-owned policy. `MAKA_CONTEXT_*` environment overrides no longer tune or disable compaction and Tool Result pruning; model-visible archive placeholders are read on demand through bounded `ArchiveRead` calls instead of eager hydration. Previously supported overrides are ignored on upgrade: if Tool Result pruning was set to `off`, pruning is re-enabled, and there is currently no supported replacement opt-out. - Moved Read image snapshots into the durable context-offload store with Runtime-owned lifecycle identity, exact branch and revision copying, recovery-safe cleanup, and bounded diff --git a/docs/architecture/llm-compaction-events-log-projection-draft.md b/docs/architecture/llm-compaction-events-log-projection-draft.md index b1aa79d215..65eff9742b 100644 --- a/docs/architecture/llm-compaction-events-log-projection-draft.md +++ b/docs/architecture/llm-compaction-events-log-projection-draft.md @@ -89,7 +89,7 @@ Next model context = Materialize( compact checkpoint, RuntimeEvents[k+1..n], provider capabilities, - current context budget + user-declared capacity and provider outcome ) ``` @@ -126,7 +126,7 @@ An ordinary summary contains only text. A safe compaction projection must also a - What is the digest of those source events? - Which high-water decision produced it? - Is it a legitimate successor to the previous checkpoint? -- Does it still fit the current token policy? +- Does its source and provider-state identity remain valid for replay? Maka therefore persists more than a string. It persists a `HistoryCompactCheckpoint`: @@ -153,7 +153,7 @@ HistoryCompactCheckpoint previousCheckpointId? ``` -For V2, the model sees `summary`. For V3, the provider sees its own opaque compact item and never the checkpoint's diagnostic text. In both cases, `coverage` determines whether the checkpoint may replace history. A projection without coverage is only a note; without a source digest it cannot establish that it still corresponds to the current log; without a replay-budget check it may be less suitable than the working set it replaces. +For V2, the model sees `summary`. For V3, the provider sees its own opaque compact item and never the checkpoint's diagnostic text. In both cases, `coverage` determines whether the checkpoint may replace history. A projection without coverage is only a note; without a source digest it cannot establish that it still corresponds to the current log; source identity and structural validation are therefore required before replay. ## Current: every request still begins with RuntimeEvents @@ -164,7 +164,7 @@ The prior-history path for a normal Send begins in `AiSdkBackend.buildPriorMessa 3. Load the latest compatible ledger-backed checkpoint. 4. Validate and replay the existing checkpoint against the immutable RuntimeEvent sequence. 5. Apply stale oversized Tool Result pruning only to the uncovered projected remainder. -6. If the projected history still exceeds the budget, select a safe prefix and retained tail. +6. When an active request has a valid user-declared Maka window and the previous accepted request's real usage crosses it, select a safe prefix and retained tail. 7. If the old checkpoint does not cover the new fold, call a compactor to create a rolling successor. 8. Validate and durably record the successor before using it. 9. Project a V2 checkpoint as a synthetic text RuntimeEvent, or carry a V3 checkpoint as explicit projection metadata, then append the uncovered raw tail. @@ -180,11 +180,11 @@ Third, the checkpoint is not itself a canonical RuntimeEvent. Coverage and tail ## Triggering ends before compaction begins -Capacity is the selected model's declared context window or nothing at all; Runtime never manufactures one. For a declared window it reserves one quarter of the window, capped at 16,384 tokens, and shapes history against the rest. Where no window is declared there is no capacity to weigh a request against, and the pre-turn gate falls back to the policy's own history-shaping budget — 32,000 tokens for most providers, and none where the provider publishes neither. Either way an estimate only asks for compaction; it never ends a request. +Capacity is the context window the user declared for the selected model, or nothing at all; Runtime never manufactures one. A provider's `/models` report and generated metadata are display hints beside the setting, not thresholds. When a declaration exists, active-turn compaction triggers only when the previous accepted request's real `inputTokens + outputTokens`, plus a reply reserve equal to the model's declared `maxOutputTokens` (zero when unknown), reaches it. With no declaration or no usable baseline there is no proactive capacity trigger. A provider `finishReason: length` or a real context-length rejection remains actionable provider evidence; whether a request fits is always the provider's decision. Trigger owners use that capacity but do not participate in compaction: -- the pre-turn and active-turn evaluators emit a Compact command when the projected request crosses the derived capacity; +- the active-turn evaluator emits a Compact command when the previous accepted request's real usage plus the reply reserve reaches the user-declared capacity, or when the provider ends the reply with `finishReason: length`; - provider-overflow recovery emits the same command after a real overflow; - manual `context.compact` emits it directly, without manufacturing a high-water crossing. @@ -197,7 +197,7 @@ trigger owner emits Compact command → append one checkpoint or leave durable state unchanged ``` -Safe-prefix selection never crosses a partial event, pinned live event, or Tool Call/Result pair. Trigger-specific callers may reserve a small verbatim successor tail, but one completed Turn remains compactable regardless of how many Agent Loop steps it contains. Context management has no environment-variable policy surface; model facts and these Runtime invariants are the only inputs. +Safe-prefix selection never crosses a partial event, pinned live event, or Tool Call/Result pair. Trigger-specific callers may reserve a small verbatim successor tail, but one completed Turn remains compactable regardless of how many Agent Loop steps it contains. Context management has no environment-variable policy surface; the user's declaration, provider usage, provider outcomes, and these Runtime invariants are the only inputs. ## What the LLM does—and does not do @@ -209,9 +209,9 @@ The LLM compactor produces a structured summary that another LLM can use to cont - Next Steps; - Critical Context, including exact paths, function names, commands, results, and errors. -The summarizer sees newly folded user/model text and Tool Calls/Results. Thinking is intentionally omitted. Runtime Host reuses the Session's selected connection, model, and provider options without imposing a compaction-only output-token cap. An output-length finish is rejected rather than admitted as a partial summary. The checkpoint builder preserves the complete accepted summary; the replay gate evaluates its full model-visible size instead of truncating it after generation. +The summarizer sees newly folded user/model text and Tool Calls/Results. Thinking is intentionally omitted. Runtime Host reuses the Session's selected connection, model, and provider options and caps compaction output at 8,000 tokens. An output-length finish is retried once with a shorter prompt and then rejected rather than admitted as a partial summary. The checkpoint builder preserves the complete accepted summary; replay validates source and provider-state identity without applying a local request-size verdict. -The text prompt and validator share one section template. A new V2 summary must contain substantive `Goal`, `Progress`, `Next Steps`, and `Critical Context` sections in order, must not end inside an open fence or other truncation marker, and must not be disproportionately small: a fold above 10,000 estimated tokens requires at least 200 estimated summary tokens. A malformed first completion gets exactly one stricter repair request, and the checkpoint write gate validates the result again. +The text prompt and validator share one section template. A new V2 summary must contain substantive `Goal`, `Progress`, `Next Steps`, and `Critical Context` sections in order, must not end inside an open fence or other truncation marker, and must not be disproportionately small: a fold whose provider reports more than 10,000 input tokens requires at least 200 reported output tokens. A malformed first completion gets exactly one stricter repair request, and the checkpoint write gate validates the result again. Malformed retries are bounded beyond that repair. Runtime remembers up to 16 exact malformed-input fingerprints per Session backend, covering the connection, model, route, policy and input budgets, request shape, previous checkpoint, and folded source events. The same unchanged input fails open without another provider dispatch; changed source or configuration is eligible again. Cancellation does not arm this circuit. Granular `malformed_summary_*` reasons survive into compaction diagnostics and terminal context-budget detail. @@ -232,11 +232,11 @@ The LLM is therefore the generator of the projection value, not the projection a When the selected connection has `providerType: openai-codex`, Maka uses Codex's server-side compactor by default instead of asking the model for a text summary. The provider request is still built from the validated RuntimeEvent prefix. The dedicated compactor sets `providerOptions.openai.compactionTrigger: true`, which appends one terminal `{ "type": "compaction_trigger" }` input item. The compactor uses the streaming Responses path and consumes the full stream because a compaction-only response has no ordinary generated-text result. Ordinary Codex requests do not set this option and are unchanged. -The portable text summarizer remains a bounded liveness fallback. Maka retries once through it when the native request receives a non-retryable protocol `RequestRejected`, returns no unique valid compact state, or cannot fit its native history projection. Cancellation, authentication, billing, rate-limit, and provider-availability failures retain their original outcome instead of doubling traffic through the same unhealthy connection. The two physical attempts share one logical compaction call but record `provider_native` and `text_summary` independently in telemetry. +The portable text summarizer remains a bounded liveness fallback. Maka retries once through it when the native request receives a non-retryable protocol `RequestRejected` or returns no unique valid compact state. Cancellation, authentication, billing, rate-limit, and provider-availability failures retain their original outcome instead of doubling traffic through the same unhealthy connection. The two physical attempts share one logical compaction call but record `provider_native` and `text_summary` independently in telemetry. Compaction input preserves assistant-step chronology. Because the Responses converter cannot resend provider-executed tool results under `store:false`, a settled hosted call/result is lowered only for this compaction request into a paired ordinary function call and output, followed by the grounded assistant text. This keeps the available tool evidence in the request without producing an orphan output. -The compaction call receives the active history-input budget. If its RuntimeEvent projection exceeds that estimate, Maka replaces older Tool Result payloads with a fixed omission marker while retaining every call/result pair and all later grounded text. If the remaining non-tool history still cannot fit, Runtime does not dispatch an already over-capacity native request and gives the text summarizer its one fallback opportunity before following the normal fail-open path. +The compaction call receives an output cap of 8,000 tokens and the provider's normal request path decides whether that call fits. Tool Result archive policies may still replace oversized individual results with durable placeholders, but history compaction does not measure the final request or reject a candidate because a local estimate says it is too large. This is deliberately a history-only contract. Maka does not send the current system prompt or tool catalog to the remote compactor, unlike the Codex CLI's whole-request assembly. Those values are neither part of checkpoint source coverage nor frozen into the checkpoint; the subsequent model request always applies its current system prompt and tools. This keeps provider-native and text-summary compactors behind the same small contract, at the cost of not giving the compactor that extra request-shape context. @@ -246,7 +246,7 @@ The V3 schema is a compatibility boundary: older binaries that only understand s ## Rolling checkpoints: do not repeatedly summarize the entire world -A long-lived Session crosses high water more than once. Resending all older events to the summarizer every time would make compaction itself increasingly expensive and repeatedly rewrite the interpretation of old facts. +A long-lived Session can create more than one rolling checkpoint. Resending all older events to the summarizer every time would make compaction itself increasingly expensive and repeatedly rewrite the interpretation of old facts. Schema V2 text checkpoints use rolling checkpoints: @@ -357,16 +357,16 @@ flowchart TD This diagram explains checkpoint-lookup recovery; it does not imply that the RuntimeEvent ledger itself needs repair. Failure to repair the bounded projection does not remove the source of an already selected checkpoint. If the canonical ledger is also unreadable, however, Runtime does not continue by guessing from a damaged cache. -## Replay: current policy judges the checkpoint again +## Replay: source and identity validate the checkpoint again -A checkpoint that was once valid is not guaranteed to fit every future request. The selected model may change, its context window may shrink, or Runtime may derive a smaller `maxHistoryEstimatedTokens` from current model facts. +A checkpoint that was once valid is not automatically valid for every future request. The selected model or connection may change, its source coverage may no longer match the ledger, or its provider-native state may no longer be replayable. -`evaluateHistoryCompactCheckpointReplay()` is the single current-policy fit gate through which a source-matched checkpoint enters model history. It recomputes the V2 model-visible checkpoint estimate (or uses the V3 estimate) and checks that: +Checkpoint replay is a source and identity gate through which a matching checkpoint enters model history. It checks that: -- checkpoint plus replay tail is within the current history budget; -- when the source projection is available for comparison, the replacement is strictly smaller than that source. +- the checkpoint is structurally valid and matches the immutable RuntimeEvent prefix; +- provider-native state belongs to the same compatible connection and model. -A projection may replay only when both source matching and current-policy fit succeed. +A projection may replay only when source matching and replay identity validation succeed. No local payload-size verdict stands between a valid projection and provider dispatch. During replay, the covered raw prefix does not enter the provider request. Uncovered folded events and retained recent events remain raw RuntimeEvents. A V2 text checkpoint produces: @@ -384,7 +384,7 @@ During replay, the covered raw prefix does not enter the provider request. Uncov The checkpoint's `limitations` state that it is only a replay-time summary of a covered RuntimeEvent prefix and that exact wording remains in the RuntimeEvent ledger. -A compatible V3 checkpoint instead produces an assistant `openai.compaction` custom part followed by the same raw tail and current Turn. Its opaque fields are never rendered as user/system text. If identity, source coverage, shape, or current-policy fit fails, Runtime keeps or recompresses the raw source-derived projection. +A compatible V3 checkpoint instead produces an assistant `openai.compaction` custom part followed by the same raw tail and current Turn. Its opaque fields are never rendered as user/system text. If identity, source coverage, or shape validation fails, Runtime keeps or recompresses the raw source-derived projection. ## Failure semantics: less context is safer than false history @@ -392,19 +392,19 @@ Compaction crosses token estimation, an LLM call, schema construction, durable a | Failure point | Current behavior | What must not happen | |---|---|---| -| Below high water | Keep the existing projection or apply ordinary budget selection | Create an unsourced summary as a speculative optimization | +| No user-declared Maka window or no usable usage baseline | Keep the source-derived projection and let the provider decide | Manufacture a Runtime capacity or estimate a first request | | LLM returns an empty summary | Record no new checkpoint. Automatic pre-turn compaction keeps the original source-derived projection and dispatches it without writing a failure note, leaving the provider to say whether it fits; manual compaction records one visible `context_compaction_failed_open` note | Treat an empty projection as covered history | | Text summary is malformed | Spend one stricter repair attempt, then fail open with a granular reason; do not redispatch an unchanged failed fingerprint | Persist incomplete structure or loop on the same doomed compaction input | | Codex returns no unique valid compact item | Try one portable text-summary checkpoint, then fail open if that also fails | Persist partial or ambiguous provider state | -| Native compaction input cannot fit after bounded Tool Result omission | Do not dispatch the native request; try one bounded text-summary checkpoint | Ask the provider to compact an already over-capacity request | -| Rolling summarizer fails | Reuse the old checkpoint if it still matches and fits, then add the newest complete raw Turns that fit | Pretend the old checkpoint covers newly evicted events | +| Provider rejects a compaction request | Follow the provider error path; do not invent a local fit verdict | Treat a local size estimate as a provider rejection | +| Rolling summarizer fails | Keep durable coverage unchanged and fail open | Pretend the old checkpoint covers newly folded events | | Durable checkpoint append fails | Do not use the candidate; fall back to the old checkpoint or safe tail | Put an uncommitted projection into the model and later claim it is recoverable | | Prefix or digest mismatch | Reject the checkpoint | Replace canonical events through approximate matching | -| Checkpoint exceeds current budget | Do not replay it | Bypass current policy because an earlier request accepted it | +| Checkpoint identity or source coverage fails | Do not replay it | Bypass source validation because an earlier request accepted it | | Bounded projection is damaged | Recover from canonical AgentRun ledgers and repair the projection | Treat the cache as the only source of truth | | User stops manual compaction | Abort the summarizer/write path without poisoning the next Turn | Persist a late result or reuse aborted state | -Fail-open here does not mean “always send the complete raw history.” Once history exceeds the model budget, the full raw prefix may itself be impossible to send. An automatic pre-turn initial V2 summary failure leaves the original source-derived projection untouched, and whether that projection still fits is the provider's answer: a rejected request is compacted and retried once, and a second rejection surfaces as a `context_overflow` provider error. Manual compaction records one visible `context_compaction_failed_open` note for the same failed outcome. A rolling failure may reuse the old checkpoint, but it never expands that checkpoint's coverage claim. +Fail-open here means “keep a valid source-derived projection and let the provider decide.” An automatic or active-turn summary failure leaves the raw projection untouched; a provider rejection is compacted and retried once, and a second rejection surfaces as a `context_overflow` provider error. Manual compaction records one visible `context_compaction_failed_open` note for the same failed outcome. A rolling failure may retain the old checkpoint, but it never expands that checkpoint's coverage claim. The correct interpretation is: @@ -485,7 +485,7 @@ Any history-compaction change must preserve these invariants: 2. **Projection coverage**: every checkpoint binds to an ordered source prefix, through boundary, and digest. 3. **No durability, no replacement**: a new checkpoint cannot replay as the accepted replacement before durable append. 4. **Monotonic high water**: a new checkpoint normally covers more events; an equal-coverage rewrite must be an explicit successor. -5. **Current-policy validation**: historical validity does not imply fitness for the current request. +5. **Current identity validation**: historical validity does not bypass current source and provider-state compatibility checks. 6. **Raw recent tail**: the model retains the newest source-derived raw context allowed by the budget. 7. **No false coverage**: a rolling failure cannot let an old summary claim new events. 8. **Projection is rebuildable**: damage to a bounded cache or projection can be repaired from canonical ledgers. @@ -502,20 +502,20 @@ First, storage does not immediately shrink when the prompt becomes shorter. Maka Second, the system must maintain coverage, digests, lineage, policy gates, recovery projections, and diagnostics. A bare summary implementation is shorter but cannot provide the same auditability. -Third, current V2 checkpoints validate source identity, shape, and budget—not the semantic completeness of the summary. Non-empty, bounded, well-structured text is not necessarily correct. A future quality gate should use source-bearing checks and record validator output as projection metadata; it should not turn the validator into a new source of truth. +Third, current V2 checkpoints validate source identity and shape—not the semantic completeness of the summary. Non-empty, bounded, well-structured text is not necessarily correct. A future quality gate should use source-bearing checks and record validator output as projection metadata; it should not turn the validator into a new source of truth. Fourth, V2 checkpoints do not currently record a complete summarizer model identity, prompt version, or request-shape hash. They are sufficient to replay an accepted projection safely, but not to promise deterministic regeneration. If Maka later needs compactor-version comparisons, offline regressions, or explanation of summary drift, those identities should enter an explicitly versioned projection manifest. -Fifth, rolling summaries can accumulate lossy error. The original log still allows a new projection to be generated from an earlier high water, but the main path currently favors incremental updates to control cost. Any future full re-compaction should be triggered by quality signals rather than an arbitrary interval. +Fifth, rolling summaries can accumulate lossy error. The original log still allows a new projection to be generated from an earlier coverage boundary, but the main path currently favors incremental updates to control cost. Any future full re-compaction should be triggered by quality signals rather than an arbitrary interval. ## Code map and verification entry points Read the current implementation from these locations: -1. `packages/runtime/src/context-budget.ts`: checkpoint-before-prune orchestration and context diagnostics; -2. `packages/runtime/src/history-compaction.ts`: high-water estimation, safe prefix/tail selection, planning, and replay policy; +1. `packages/runtime/src/context-budget.ts`: durable projection orchestration and context diagnostics; +2. `packages/runtime/src/history-compaction.ts`: safe prefix/tail selection, planning, and replay policy; 3. `packages/runtime/src/history-compact-checkpoint.ts`: V2/V3 schemas, provider identity, digest, prefix match, lineage, and replay materialization; -4. `packages/runtime/src/history-compact-summary-validation.ts`: the shared section, truncation, and large-fold size gates; +4. `packages/runtime/src/history-compact-summary-validation.ts`: the shared section, truncation, and provider-usage quality gates; 5. `packages/runtime/src/history-compact-summarizer.ts`: LLM continuation-summary prompt, bounded repair, and rolling input; 6. `packages/runtime/src/ai-sdk-compaction.ts`: compaction orchestration, malformed-input circuit, writes, and fallback semantics; 7. `packages/runtime/src/ai-sdk-backend.ts`: prior-history request projection and provider materialization; @@ -523,15 +523,15 @@ Read the current implementation from these locations: 9. `packages/runtime/src/history-compact-ledger.ts`: bounded-projection lookup, ledger recovery, and checkpoint selection; 10. `packages/runtime/src/runtime-kernel.ts`: serialized checkpoint writes and manual-compaction lifecycle; 11. `packages/storage/src/agent-run-store.ts`: atomic canonical-event and bounded-projection persistence; -12. `packages/runtime/src/context-budget-policy.ts`: model-capacity derivation and fixed Runtime policy; +12. `packages/runtime/src/context-budget-policy.ts`: user-declared capacity resolution and fixed Runtime policy; 13. `packages/runtime/src/openai-codex-history-compactor.ts`: Codex compact-output validation and rolling provider-state input; 14. `packages/runtime-host/src/server/execution-model-composition.ts`: default provider-specific compactor selection. Important tests include: -- `history-compact-checkpoint.test.ts`: coverage metadata, prefix digest, summary admission, ledger recovery, projection repair, and policy replay; -- `history-compaction.test.ts`: high-water estimation, safe prefix/tail selection, Tool pair preservation, rolling updates, and write gates; -- `history-compact-summarizer.test.ts`: provider options, input fitting, structured-summary validation and repair, and rolling input; +- `history-compact-checkpoint.test.ts`: coverage metadata, prefix digest, summary admission, ledger recovery, projection repair, and replay; +- `history-compaction.test.ts`: safe prefix/tail selection, Tool pair preservation, rolling updates, and write gates; +- `history-compact-summarizer.test.ts`: provider options, output limits, structured-summary validation and repair, and rolling input; - `context-budget.test.ts`: canonical-ledger retention and checkpoint replay before stale Tool Result pruning; - `context-budget-mid-turn-policy.test.ts`: model-capacity derivation and fixed Runtime defaults; - `mid-turn-capacity-backend.test.ts`: persist-before-apply, fail-open/exhaustion detail, and active-turn retry bounds; @@ -546,16 +546,16 @@ Maka's LLM compaction is not a destructive rewrite of a conversation table. It i ```text RuntimeEvent prefix - → deterministic coverage and high-water selection + → deterministic coverage selection → LLM continuation summary → durable HistoryCompactCheckpoint event - → source/digest/current-policy validation + → source/digest/provider-state identity validation → synthetic checkpoint RuntimeEvent + raw recent tail → provider-specific ModelMessage projection ``` The elegant part is not that an LLM can write a good summary. It is that the system never mistakes that summary for history itself. -The log answers “what happened.” The checkpoint answers “from this high water, how may the next inference continue?” The provider request answers “what does this model need to see for this call?” Each has a different lifecycle and explicit authority. +The log answers “what happened.” The checkpoint answers “from this coverage boundary, how may the next inference continue?” The provider request answers “what does this model need to see for this call?” Each has a different lifecycle and explicit authority. -Therefore, **compaction is the Events Log's projection** is more than a design slogan. It means the source cannot be overwritten by a summary, a projection must carry coverage, accepted replacement must be durable, replay must pass current policy again, and every projection must be disposable, verifiable, or rebuildable from the log. +Therefore, **compaction is the Events Log's projection** is more than a design slogan. It means the source cannot be overwritten by a summary, a projection must carry coverage, accepted replacement must be durable, replay must pass source and provider-state identity checks, and every projection must be disposable, verifiable, or rebuildable from the log. diff --git a/docs/architecture/llm-compaction-events-log-projection-draft.zh-CN.md b/docs/architecture/llm-compaction-events-log-projection-draft.zh-CN.md index 5239350a6d..b8ab2b1a25 100644 --- a/docs/architecture/llm-compaction-events-log-projection-draft.zh-CN.md +++ b/docs/architecture/llm-compaction-events-log-projection-draft.zh-CN.md @@ -89,7 +89,7 @@ Next model context = Materialize( compact checkpoint, RuntimeEvents[k+1..n], provider capabilities, - current context budget + 用户声明的 capacity 与 provider 结果 ) ``` @@ -126,7 +126,7 @@ flowchart LR - 这些源事件的 digest 是什么？ - 它由哪一个 high-water decision 产生？ - 它是否是上一 checkpoint 的合法 successor？ -- 它在当前 token policy 下仍然能否进入 prompt？ +- 它的 source 与 provider-state identity 在 replay 时是否仍然有效？ 所以，Maka 持久化的不是一个裸字符串，而是 `HistoryCompactCheckpoint`： @@ -153,7 +153,7 @@ HistoryCompactCheckpoint previousCheckpointId? ``` -V2 中模型主要看到 `summary`；V3 中 provider 看到自己的 opaque compact item，不会看到 checkpoint 的诊断文本。两者都由 `coverage` 决定有没有资格替代历史。没有 coverage 的 projection 只是笔记；没有 source digest 就无法证明它仍对应当前日志；没有 replay budget 校验，它可能比被替代的工作集更不适合当前请求。 +V2 中模型主要看到 `summary`；V3 中 provider 看到自己的 opaque compact item，不会看到 checkpoint 的诊断文本。两者都由 `coverage` 决定有没有资格替代历史。没有 coverage 的 projection 只是笔记；没有 source digest 就无法证明它仍对应当前日志；因此 replay 前必须通过 source identity 与结构校验。 ## Current：完整请求仍从 RuntimeEvents 开始 @@ -164,7 +164,7 @@ V2 中模型主要看到 `summary`；V3 中 provider 看到自己的 opaque comp 3. 加载最新且兼容的 ledger-backed checkpoint； 4. 在 immutable RuntimeEvent 序列上校验并 replay 已有 checkpoint； 5. 只对未覆盖的 projected remainder 执行 stale oversized Tool Result prune； -6. 如果 projected history 仍超出预算，选择 safe prefix 与 retained tail； +6. 如果 active request 有用户声明的 Maka 窗口，且上一次成功请求的真实 usage 加上回复预留（模型声明的 `maxOutputTokens`，未知时为 0）达到它，选择 safe prefix 与 retained tail； 7. 如果旧 checkpoint 不足以覆盖新的 fold，调用 compactor 滚动生成 successor； 8. successor 通过校验并 durable record 后才能使用； 9. V2 checkpoint 投影为 synthetic text RuntimeEvent；V3 checkpoint 则作为显式 projection metadata 传递，然后拼接未覆盖 raw tail； @@ -180,11 +180,11 @@ V2 中模型主要看到 `summary`；V3 中 provider 看到自己的 opaque comp ## Trigger 在 compaction 开始前结束 -Capacity 要么是所选模型声明的 context window，要么根本不存在，Runtime 不会自己造一个。有声明的 window 时，reserve 取 window 的四分之一且上限为 16,384 tokens，其余部分用来塑形历史；没有声明 window 时就没有可用来衡量请求的 capacity，pre-turn gate 退回到 policy 自己的 history-shaping budget——多数 provider 是 32,000 tokens，两者都不公布的 provider 则没有预算。无论哪种情况，估算都只用来请求 compaction，不会结束请求。 +Capacity 是用户为所选模型声明的 context window，否则就不存在；Runtime 不会自己造一个。provider 的 `/models` 报告和生成 metadata 只在设置项旁作为提示，不是阈值。有声明时，active-turn compaction 只在上一次成功请求的真实 `inputTokens + outputTokens` 加上回复预留（模型声明的 `maxOutputTokens`，未知时为 0）达到它时触发。没有声明或没有可用 baseline 时，不主动按 capacity 折叠。provider 的 `finishReason: length` 或真实 context-length rejection 仍然是 provider 证据；请求是否放得下始终由 provider 决定。 Trigger owner 使用这个 capacity，但不参与 compaction： -- pre-turn 与 active-turn evaluator 在 projected request 越过推导出的 capacity 时发出 Compact command； +- active-turn evaluator 在上一次成功请求的真实 usage 加上回复预留达到用户声明的 capacity，或 provider 以 `finishReason: length` 结束回复时发出 Compact command； - provider-overflow recovery 在真实 overflow 后发出同一 command； - 手动 `context.compact` 直接发出 command，不伪造 high-water crossing。 @@ -197,7 +197,7 @@ trigger owner emits Compact command → append one checkpoint or leave durable state unchanged ``` -Safe-prefix selection 不会跨越 partial event、pinned live event 或 Tool Call/Result pair。Trigger-specific caller 可以保留一小段 verbatim successor tail，但一个 completed Turn 无论包含多少 Agent Loop steps 都可以 compact。Context management 不再提供环境变量 policy surface；模型事实与这些 Runtime invariant 是唯一输入。 +Safe-prefix selection 不会跨越 partial event、pinned live event 或 Tool Call/Result pair。Trigger-specific caller 可以保留一小段 verbatim successor tail，但一个 completed Turn 无论包含多少 Agent Loop steps 都可以 compact。Context management 不再提供环境变量 policy surface；用户声明、provider usage、provider outcome 与这些 Runtime invariant 是唯一输入。 ## LLM 在这里做什么，也不做什么 @@ -209,9 +209,9 @@ LLM compaction 的任务是生成一份“让另一个 LLM 继续工作”的结 - Next Steps； - Critical Context，包括精确路径、函数名、命令、结果和错误。 -Summarizer 会看到被新折叠的用户/模型文本与 tool call/result。Thinking 被有意排除。Runtime Host 复用当前 Session 选中的 connection、model 与 provider options，不额外设置 compaction-only output-token cap。如果 provider 以 output-length 结束，这份不完整 summary 会被拒绝。Checkpoint builder 保留完整的已接受 summary；replay gate 按完整 model-visible size 判断，而不是在生成后截断。 +Summarizer 会看到被新折叠的用户/模型文本与 tool call/result。Thinking 被有意排除。Runtime Host 复用当前 Session 选中的 connection、model 与 provider options，并把 compaction output 限制为 8,000 tokens。若 provider 以 output-length 结束，先用更短 prompt 重试一次，仍然不完整的 summary 会被拒绝。Checkpoint builder 保留完整的已接受 summary；replay 校验 source 与 provider-state identity，不再应用本地 request-size verdict。 -文本 prompt 与 validator 共用一份 section template。新的 V2 summary 必须依次包含有实质内容的 `Goal`、`Progress`、`Next Steps` 与 `Critical Context`，不能结束在未闭合 fence 或其他 truncation marker 上，也不能相对 fold 过小：fold 超过 10,000 estimated tokens 时，summary 至少需要 200 estimated tokens。第一份 malformed completion 只有一次更严格的 repair request，checkpoint write gate 随后还会再次校验结果。 +文本 prompt 与 validator 共用一份 section template。新的 V2 summary 必须依次包含有实质内容的 `Goal`、`Progress`、`Next Steps` 与 `Critical Context`，不能结束在未闭合 fence 或其他 truncation marker 上，也不能相对 fold 过小：provider 报告 fold input 超过 10,000 tokens 时，summary 至少需要 200 个 provider output tokens。第一份 malformed completion 只有一次更严格的 repair request，checkpoint write gate 随后还会再次校验结果。 Repair 之外的 malformed retry 也有上限。Runtime 为每个 Session backend 最多记住 16 个精确 malformed-input fingerprint，其中覆盖 connection、model、route、policy 与 input budget、request shape、previous checkpoint 和 folded source events。输入不变时直接 fail open，不再 dispatch provider；source 或配置变化后可以重试。Cancellation 不会触发这个 circuit。细分的 `malformed_summary_*` reason 会一直保留到 compaction diagnostics 与 terminal context-budget detail。 @@ -232,11 +232,11 @@ Repair 之外的 malformed retry 也有上限。Runtime 为每个 Session backen 当所选 connection 的 `providerType` 为 `openai-codex` 时，Maka 默认使用 Codex 服务端 compactor，不再让模型生成文本摘要。provider request 仍由已校验的 RuntimeEvent prefix 构建。专用 compactor 会设置 `providerOptions.openai.compactionTrigger: true`，从而追加唯一一个位于 input 末尾的 `{ "type": "compaction_trigger" }` item。compactor 使用流式 Responses 路径并消费完整 stream，因为只有 compaction output 的响应不存在普通 generated-text result；普通 Codex 请求不会设置这个选项，因此行为不变。 -可移植的文本 summarizer 仍作为有界的 liveness fallback。当 native request 收到不可重试的协议级 `RequestRejected`、没有返回唯一合法的 compact state，或无法容纳 native history projection 时，Maka 会通过文本 summarizer 重试一次。Cancellation、鉴权、计费、限流和 provider 不可用仍保留原始结果，不会向同一个异常连接发送双倍流量。两次物理请求属于同一个逻辑 compaction call，但 telemetry 会分别记录 `provider_native` 与 `text_summary`。 +可移植的文本 summarizer 仍作为有界的 liveness fallback。当 native request 收到不可重试的协议级 `RequestRejected` 或没有返回唯一合法的 compact state 时，Maka 会通过文本 summarizer 重试一次。Cancellation、鉴权、计费、限流和 provider 不可用仍保留原始结果，不会向同一个异常连接发送双倍流量。两次物理请求属于同一个逻辑 compaction call，但 telemetry 会分别记录 `provider_native` 与 `text_summary`。 Compaction input 会保留 assistant step 的时序。由于 Responses converter 在 `store:false` 下无法重新发送 provider-executed tool result，已经完整结算的 hosted call/result 只在这次 compaction request 中降级为成对的普通 function call 与 output，之后再放 grounded assistant text。这样既保留了现有 tool evidence，也不会生成悬空 output。 -Compaction call 会收到当前 history input budget。若 RuntimeEvent projection 超出该估算值，Maka 会把较旧的 Tool Result payload 替换为固定 omission marker，同时保留每一组 call/result 配对和之后的 grounded text。若剩余的非工具历史仍无法容纳，Runtime 不会发送一条已经超出容量的 native request，而是先给文本 summarizer 一次 fallback 机会，再进入正常 fail-open 路径。 +Compaction call 的 output 上限是 8,000 tokens，是否能放下由 provider 的正常请求路径决定。Tool Result archive policy 仍可以把过大的单条结果替换为可持久化 placeholder，但 history compaction 不再量最终 request，也不会因为本地估算认为过大而拒绝候选。 这是有意设计成 history-only 的契约。与 Codex CLI 的 whole-request assembly 不同，Maka 不会把当前 system prompt 或 tool catalog 发给 remote compactor；它们既不属于 checkpoint source coverage，也不会被冻结进 checkpoint，后续模型请求始终使用当时最新的 system prompt 和 tools。这样 provider-native 与 text-summary compactor 可以共享同一份小契约，代价是 compactor 无法利用这部分额外的 request-shape context。 @@ -246,7 +246,7 @@ V3 schema 也是兼容边界：只理解 schema V2 的旧 binary 会拒绝它并 ## Rolling checkpoint：不要反复总结整个世界 -长期 Session 会多次越过 high water。如果每次都把所有旧事件重新发送给 summarizer，compaction 自身会变成越来越昂贵的请求，也会让旧事实被反复改写。 +长期 Session 会创建多个 rolling checkpoint。如果每次都把所有旧事件重新发送给 summarizer，compaction 自身会变成越来越昂贵的请求，也会让旧事实被反复改写。 Schema V2 文本 checkpoint 使用 rolling checkpoint： @@ -357,16 +357,16 @@ flowchart TD 这张图解释 checkpoint lookup 的恢复关系，不代表 RuntimeEvent ledger 本身需要修复。Projection repair 失败不会让已经选出的 checkpoint 失去来源；但如果 canonical ledger 也无法读取，系统不会凭损坏缓存继续猜测。 -## Replay：Checkpoint 必须再次接受当前 policy 审判 +## Replay：Checkpoint 必须再次接受 source 与 identity 校验 -一个曾经合法的 checkpoint 不保证永远适合所有请求。所选模型可能切换，context window 可能变小，Runtime 也可能根据当前 model facts 推导出更小的 `maxHistoryEstimatedTokens`。 +一个曾经合法的 checkpoint 不会自动适用于所有后续请求。所选 model 或 connection 可能切换，source coverage 可能不再匹配日志，provider-native state 也可能不再可 replay。 -`evaluateHistoryCompactCheckpointReplay()` 是 source-matched checkpoint 进入模型历史的统一 current-policy fit gate。它重新计算 V2 model-visible checkpoint estimate（V3 使用已记录的 estimate），并检查： +Checkpoint replay 是 source 与 identity gate。只有匹配的 checkpoint 才能进入模型历史，并检查： -- checkpoint 与 replay tail 合计不超过当前 history budget； -- 如果有 source projection 可供比较，replacement 必须严格小于该 source。 +- checkpoint 结构有效，并且匹配不可变的 RuntimeEvent prefix； +- provider-native state 属于兼容的同一 connection 与 model。 -只有 source match 与 current-policy fit 同时成立，projection 才能 replay。 +只有 source match 与 replay identity validation 同时成立，projection 才能 replay。本地 payload-size verdict 不会挡在有效 projection 与 provider dispatch 之间。 Replay 时，covered raw prefix 不进入 provider request；未覆盖的 folded suffix 与 retained recent events 继续以 raw RuntimeEvents 存在。V2 文本 checkpoint 会让模型看到： @@ -384,7 +384,7 @@ Replay 时，covered raw prefix 不进入 provider request；未覆盖的 folded Checkpoint 的 `limitations` 会明确提醒：它只是 covered RuntimeEvent prefix 的 replay-time summary；精确措辞仍应回到 RuntimeEvent ledger。 -兼容的 V3 checkpoint 则产生 assistant `openai.compaction` custom part，后面拼接同一份 raw tail 与 current Turn；opaque 字段绝不会被渲染为 user/system 文本。identity、source coverage、shape 或 current-policy fit 任一失败时，Runtime 会保留或重新压缩 source-derived raw projection。 +兼容的 V3 checkpoint 则产生 assistant `openai.compaction` custom part，后面拼接同一份 raw tail 与 current Turn；opaque 字段绝不会被渲染为 user/system 文本。identity、source coverage 或 shape 任一失败时，Runtime 会保留或重新压缩 source-derived raw projection。 ## Failure semantics：宁可少看，也不要看一份假历史 @@ -392,19 +392,19 @@ Compaction 跨越 token estimation、LLM call、schema construction、durable ap | 失败位置 | 当前行为 | 不允许发生的事 | |---|---|---| -| 未超过 high water | 保持原投影或普通预算裁剪 | 为了“提前优化”制造无来源摘要 | +| 没有用户声明的 Maka window 或没有可用 usage baseline | 保留 source-derived projection，由 provider 裁决 | Runtime 自造 capacity 或估算第一次请求 | | LLM 返回空 summary | 不记录新 checkpoint。自动 pre-turn compaction 保留原有的 source-derived projection 并照常发出，不写入失败 note，放不放得下由 provider 回答；手动 compaction 则记录一次可见的 `context_compaction_failed_open` note | 把空 projection 当作 covered history | | Text summary 格式不合法 | 只进行一次更严格的 repair，之后以细分 reason fail open；同一失败 fingerprint 不再 dispatch | 持久化不完整结构，或在相同 doomed input 上循环 | | Codex 没有返回唯一且合法的 compact item | 尝试一次可移植文本摘要 checkpoint；若仍失败再 fail open | 持久化残缺或有歧义的 provider state | -| Native compaction input 在有界省略 Tool Result 后仍无法容纳 | 不发送 native request，尝试一次有界文本摘要 checkpoint | 要求 provider 压缩一条已经超出容量的请求 | -| Rolling summarizer 失败 | 若旧 checkpoint 仍匹配且符合当前限制，则复用它并拼接能容纳的最新完整 raw Turns | 假装旧 checkpoint 已覆盖 newly evicted events | +| Provider 拒绝 compaction request | 遵循 provider error path，不编造本地 fit verdict | 把本地大小估算当作 provider rejection | +| Rolling summarizer 失败 | 保持 durable coverage 不变并 fail open | 假装旧 checkpoint 已覆盖新折叠的 events | | Durable checkpoint append 失败 | 不使用 candidate；回退旧 checkpoint 或安全 tail | 让未提交 projection 进入模型后再声称可恢复 | | Prefix 或 digest 不匹配 | 拒绝 checkpoint | 用近似匹配替代 canonical events | -| Checkpoint 超出当前 budget | 不 replay 它 | 因为过去接受过就绕过当前 policy | +| Checkpoint identity 或 source coverage 失败 | 不 replay 它 | 因为过去接受过就绕过 source validation | | Bounded projection 损坏 | 从 canonical AgentRun ledger 恢复并修复 projection | 把缓存当成唯一事实源 | | 用户停止 manual compaction | 中止 summarizer/write 链路，不污染下一 Turn | 让迟到结果写入或复用 abort state | -这里的 fail-open 不是“无论如何发送完整历史”。当历史已经超过模型预算时，完整 raw prefix 本身可能不可发送。自动 pre-turn 的 V2 初次 summary 失败会原样保留 source-derived projection；该 projection 放不放得下由 provider 决定：请求被拒后压缩并重试一次，再次被拒则以 `context_overflow` 报 provider 错误。手动 compaction 对同一失败结果写入一次可见的 `context_compaction_failed_open` note。Rolling failure 可以复用旧 checkpoint，但绝不会扩大它的 coverage claim。 +这里的 fail-open 意味着“保留有效的 source-derived projection，由 provider 裁决”。automatic 或 active-turn summary 失败时保留 raw projection；provider rejection 会触发一次折叠重试，第二次 rejection 则作为 `context_overflow` provider error 暴露。manual compaction 对同样的失败结果记录一条可见的 `context_compaction_failed_open` note。rolling failure 可以保留旧 checkpoint，但不会扩大其 coverage claim。 正确理解是： @@ -485,7 +485,7 @@ LLM 在 summary 中写“测试已通过”仍然只是对 source events 的概 2. **Projection coverage**：每个 checkpoint 都绑定一个有序 source prefix、through boundary 和 digest。 3. **No durability, no replacement**：新 checkpoint 未 durable append 时，不得作为 accepted replacement replay。 4. **Monotonic high water**：新 checkpoint 通常必须覆盖更多 events；同 coverage rewrite 必须是显式 successor。 -5. **Current-policy validation**：历史上合法不代表当前 request 可以使用。 +5. **Current identity validation**：历史上合法不代表绕过当前 source 与 provider-state 兼容性检查。 6. **Raw recent tail**：模型始终获得当前预算允许的最新 source-derived raw context。 7. **No false coverage**：rolling failure 不得让旧 summary 声称覆盖新 events。 8. **Projection is rebuildable**：bounded cache/projection 损坏时，可以从 canonical ledger 恢复。 @@ -502,20 +502,20 @@ LLM 在 summary 中写“测试已通过”仍然只是对 source events 的概 第二，系统需要维护 coverage、digest、lineage、policy gate、recovery projection 与 diagnostics。一个裸 summary 实现更短，但无法提供相同的可审计性。 -第三，当前 V2 checkpoint 只验证 source identity、shape 与预算，不验证 summary 的语义完备性。非空、bounded、结构清晰不等于内容正确。未来如果引入 summary quality gate，应当使用 source-bearing checks，并把 validator 结果作为 projection metadata，而不是把 validator 变成新的事实源。 +第三，当前 V2 checkpoint 只验证 source identity 与 shape，不验证 summary 的语义完备性。非空、bounded、结构清晰不等于内容正确。未来如果引入 summary quality gate，应当使用 source-bearing checks，并把 validator 结果作为 projection metadata，而不是把 validator 变成新的事实源。 第四，V2 checkpoint 当前没有完整记录 summarizer model identity、prompt version 或 request-shape hash。它足以安全 replay 已接受 projection，却不足以承诺确定性再生成。若未来需要比较 compactor 版本、做离线回归或解释摘要漂移，这些字段值得进入明确版本化的 projection manifest。 -第五，rolling summary 会积累有损误差。原始日志仍然允许重新从更早 high water 生成新 projection，但当前主路径优先增量更新以控制成本。什么时候触发 full re-compaction，应由质量信号而不是任意时间间隔决定。 +第五，rolling summary 会积累有损误差。原始日志仍然允许重新从更早 coverage boundary 生成新 projection，但当前主路径优先增量更新以控制成本。什么时候触发 full re-compaction，应由质量信号而不是任意时间间隔决定。 ## 代码地图与验证入口 当前实现可以从以下位置阅读： -1. `packages/runtime/src/context-budget.ts`：checkpoint-before-prune orchestration 与 context diagnostics； -2. `packages/runtime/src/history-compaction.ts`：high-water estimation、safe prefix/tail selection、planning 与 replay policy； +1. `packages/runtime/src/context-budget.ts`：durable projection orchestration 与 context diagnostics； +2. `packages/runtime/src/history-compaction.ts`：safe prefix/tail selection、planning 与 replay policy； 3. `packages/runtime/src/history-compact-checkpoint.ts`：V2/V3 schema、provider identity、digest、prefix match、lineage 与 replay materialization； -4. `packages/runtime/src/history-compact-summary-validation.ts`：共用的 section、truncation 与 large-fold size gate； +4. `packages/runtime/src/history-compact-summary-validation.ts`：共用的 section、truncation 与 provider-usage quality gate； 5. `packages/runtime/src/history-compact-summarizer.ts`：LLM continuation-summary prompt、bounded repair 与 rolling input； 6. `packages/runtime/src/ai-sdk-compaction.ts`：compaction orchestration、malformed-input circuit、write 与 fallback 语义； 7. `packages/runtime/src/ai-sdk-backend.ts`：prior-history request projection 与 provider materialization； @@ -523,15 +523,15 @@ LLM 在 summary 中写“测试已通过”仍然只是对 source events 的概 9. `packages/runtime/src/history-compact-ledger.ts`：bounded projection lookup、ledger recovery 与 checkpoint selection； 10. `packages/runtime/src/runtime-kernel.ts`：checkpoint write serialization 与 manual compaction lifecycle； 11. `packages/storage/src/agent-run-store.ts`：canonical event 与 bounded projection 的 atomic persistence； -12. `packages/runtime/src/context-budget-policy.ts`：model-capacity derivation 与固定 Runtime policy； +12. `packages/runtime/src/context-budget-policy.ts`：user-declared capacity resolution 与固定 Runtime policy； 13. `packages/runtime/src/openai-codex-history-compactor.ts`：Codex compact output 校验与 rolling provider-state input； 14. `packages/runtime-host/src/server/execution-model-composition.ts`：默认 provider-specific compactor 选择。 重点测试包括： - `history-compact-checkpoint.test.ts`：coverage metadata、prefix digest、summary admission、ledger recovery、projection repair 与 policy replay； -- `history-compaction.test.ts`：high-water estimation、safe prefix/tail selection、Tool pair preservation、rolling update 与 write gate； -- `history-compact-summarizer.test.ts`：provider options、input fitting、structured-summary validation/repair 与 rolling input； +- `history-compaction.test.ts`：safe prefix/tail selection、Tool pair preservation、rolling update 与 write gate； +- `history-compact-summarizer.test.ts`：provider options、output limits、structured-summary validation/repair 与 rolling input； - `context-budget.test.ts`：canonical-ledger retention，以及 checkpoint 在 stale Tool Result prune 前 replay； - `context-budget-mid-turn-policy.test.ts`：model-capacity derivation 与固定 Runtime defaults； - `mid-turn-capacity-backend.test.ts`：persist-before-apply、fail-open/exhaustion detail 与 active-turn retry bound； @@ -546,16 +546,16 @@ Maka 的 LLM compaction 不是一次对 conversation table 的 destructive rewri ```text RuntimeEvent prefix - → deterministic coverage and high-water selection + → deterministic coverage selection → LLM continuation summary → durable HistoryCompactCheckpoint event - → source/digest/current-policy validation + → source/digest/provider-state identity validation → synthetic checkpoint RuntimeEvent + raw recent tail → provider-specific ModelMessage projection ``` 这条链的精妙之处不在于 LLM 能写出多漂亮的摘要，而在于系统从未把摘要误认为历史本身。 -日志回答“发生过什么”；checkpoint 回答“在这个 high water 上，下一次推理可以怎样继续”；provider request 回答“这个模型在这一次调用里实际需要看到什么”。三者各自有不同生命周期，也各自有明确的 authority。 +日志回答“发生过什么”；checkpoint 回答“在这个 coverage boundary 上，下一次推理可以怎样继续”；provider request 回答“这个模型在这一次调用里实际需要看到什么”。三者各自有不同生命周期，也各自有明确的 authority。 -所以，**compaction is the Events Log's projection** 不只是一句设计口号。它具体意味着：source 不可被摘要覆盖，projection 必须带 coverage，accepted replacement 必须 durable，replay 必须重新通过当前 policy，而任何 projection 都应该能够被丢弃、校验或从日志重建。 +所以，**compaction is the Events Log's projection** 不只是一句设计口号。它具体意味着：source 不可被摘要覆盖，projection 必须带 coverage，accepted replacement 必须 durable，replay 必须重新通过 source 与 provider-state identity 校验，而任何 projection 都应该能够被丢弃、校验或从日志重建。 diff --git a/packages/cli/src/pi-transcript.ts b/packages/cli/src/pi-transcript.ts index 9374fbfbb4..ddee823da5 100644 --- a/packages/cli/src/pi-transcript.ts +++ b/packages/cli/src/pi-transcript.ts @@ -1345,6 +1345,45 @@ function systemNoteText(message: SystemNoteMessage): string | undefined { return 'Context compacted to keep this task within the model window.'; case 'context_compaction_failed_open': return 'Context summary failed; the session continued without a new summary.'; + case 'context_provider_dropping': + return 'The provider is dropping or rewriting context: content was appended but its reported usage did not grow. Declare a context window for this model so Maka compacts first.'; + case 'context_reported_window_exceeded': { + const data = message.data as + | { usedTokens?: unknown; reportedContextWindow?: unknown } + | undefined; + const used = typeof data?.usedTokens === 'number' ? data.usedTokens : undefined; + const reported = + typeof data?.reportedContextWindow === 'number' ? data.reportedContextWindow : undefined; + if (used === undefined || reported === undefined) { + return 'This exchange ran past the context window this model reports, and the provider accepted it anyway.'; + } + return `This exchange used about ${used} tokens, past the ${reported} this model reports, and the provider accepted it without complaint. Nothing is declared, so Maka does not compact on its own; declare a context window to have it compact first.`; + } + case 'context_window_overrun': { + const data = message.data as + | { usedTokens?: unknown; declaredContextWindow?: unknown } + | undefined; + const used = typeof data?.usedTokens === 'number' ? data.usedTokens : undefined; + const declared = + typeof data?.declaredContextWindow === 'number' ? data.declaredContextWindow : undefined; + if (used === undefined || declared === undefined) { + return 'This exchange ran past the context window declared for this model.'; + } + return `This exchange used about ${used} tokens against the declared window of ${declared}: the reply needed more room than was left. Maka compacts before the next request; raise the window if the replies should stay whole.`; + } + case 'context_window_suggestion': { + const data = message.data as + | { suggestedContextWindow?: unknown; declaredContextWindow?: unknown } + | undefined; + const tokens = + typeof data?.suggestedContextWindow === 'number' ? data.suggestedContextWindow : undefined; + const declared = + typeof data?.declaredContextWindow === 'number' ? data.declaredContextWindow : undefined; + if (tokens === undefined) return 'The provider rejected this request as too large.'; + return declared === undefined + ? `The provider rejected this request. No context window is declared for this model; the last accepted request was about ${tokens} tokens — declare that as the window so Maka compacts first.` + : `The provider rejected this request at about ${tokens} tokens, below the declared window of ${declared}. The declaration is likely larger than the provider's window; consider lowering it to ${tokens}.`; + } case 'step_limit': return STEP_LIMIT_NOTICE_TEXT; case 'error': diff --git a/packages/core/src/__tests__/usage-record-last-request-anchor.test.ts b/packages/core/src/__tests__/usage-record-last-request-anchor.test.ts index 659169ae0d..c8aba6ee6e 100644 --- a/packages/core/src/__tests__/usage-record-last-request-anchor.test.ts +++ b/packages/core/src/__tests__/usage-record-last-request-anchor.test.ts @@ -24,28 +24,33 @@ import { decodeCanonicalMessage } from '../session.js'; const usage = { input: 370, output: 60 }; -test('a last-request anchor is only valid as a complete positive pair', () => { +test('a last-request anchor accepts the new usage shape and retired payload key', () => { + assert.equal(isLastRequestAnchor({ inputTokens: 120, outputTokens: 30 }), true); + assert.equal(isLastRequestAnchor({ inputTokens: 120 }), true); assert.equal(isLastRequestAnchor({ inputTokens: 120, payloadChars: 4_000 }), true); - assert.equal(isLastRequestAnchor({ inputTokens: 120 }), false); assert.equal(isLastRequestAnchor({ payloadChars: 4_000 }), false); assert.equal(isLastRequestAnchor({ inputTokens: 0, payloadChars: 4_000 }), false); - assert.equal(isLastRequestAnchor({ inputTokens: 120, payloadChars: 0 }), false); - assert.equal( - isLastRequestAnchor({ inputTokens: 120, payloadChars: 4_000, stepNumber: 2 }), - false, - ); + assert.equal(isLastRequestAnchor({ inputTokens: 120, outputTokens: -1 }), false); + assert.equal(isLastRequestAnchor({ inputTokens: 120, foo: 1 }), false); }); test('token-usage fields carry the anchor and reject a broken one', () => { assert.equal( - isTokenUsageFields({ ...usage, lastRequestAnchor: { inputTokens: 120, payloadChars: 4_000 } }), + isTokenUsageFields({ ...usage, lastRequestAnchor: { inputTokens: 120, outputTokens: 30 } }), true, ); assert.equal(isTokenUsageFields(usage), true); - assert.equal(isTokenUsageFields({ ...usage, lastRequestAnchor: { inputTokens: 120 } }), false); + assert.equal( + isTokenUsageFields({ ...usage, lastRequestAnchor: { inputTokens: 120, payloadChars: 4_000 } }), + true, + ); + assert.equal( + isTokenUsageFields({ ...usage, lastRequestAnchor: { inputTokens: 120, foo: 1 } }), + false, + ); }); -test('a half-written anchor fails the whole token_usage message decode', () => { +test('an invalid anchor fails the whole token_usage message decode', () => { const message = { type: 'token_usage', id: 'usage-1', @@ -56,11 +61,11 @@ test('a half-written anchor fails the whole token_usage message decode', () => { assert.deepEqual( decodeCanonicalMessage({ ...message, - lastRequestAnchor: { inputTokens: 120, payloadChars: 4_000 }, + lastRequestAnchor: { inputTokens: 120, outputTokens: 30 }, }), - { ...message, lastRequestAnchor: { inputTokens: 120, payloadChars: 4_000 } }, + { ...message, lastRequestAnchor: { inputTokens: 120, outputTokens: 30 } }, ); assert.throws(() => - decodeCanonicalMessage({ ...message, lastRequestAnchor: { payloadChars: 4_000 } }), + decodeCanonicalMessage({ ...message, lastRequestAnchor: { inputTokens: 0 } }), ); }); diff --git a/packages/core/src/model-thinking.ts b/packages/core/src/model-thinking.ts index 2abbc40032..177ae11188 100644 --- a/packages/core/src/model-thinking.ts +++ b/packages/core/src/model-thinking.ts @@ -268,6 +268,39 @@ export function relayModelProfile( return normalizeRelayModelProfile(connection.relayModelProfiles?.[modelId]); } +/** The connection fields the declared-window rule reads; structural so runtime and UI projections both fit. */ +export interface DeclaredContextWindowContext extends ConnectionThinkingContext { + readonly models?: readonly { + readonly id: string; + readonly contextWindow?: number; + readonly inputLimit?: number; + readonly factOverriddenFields?: readonly string[]; + }[]; +} + +/** + * The context window the USER declared for a model — the Maka window: the + * proactive compaction target, and nothing else. Exactly two sources count as + * a declaration: a model-facts pin (`factOverriddenFields` includes + * `contextWindow`, narrowest of window/input limit) and a relay model profile. + * A provider's `/models` report and generated metadata describe the model and + * are shown as a hint; they never become a threshold on their own. This is the + * single owner of that rule for runtime and UI (#4559). + */ +export function declaredContextWindow( + connection: DeclaredContextWindowContext, + modelId: string, +): number | undefined { + const model = connection.models?.find((candidate) => candidate.id === modelId); + if (model?.factOverriddenFields?.includes('contextWindow')) { + const values = [model.contextWindow, model.inputLimit].filter( + (value): value is number => typeof value === 'number' && Number.isFinite(value) && value > 0, + ); + return values.length > 0 ? Math.min(...values) : undefined; + } + return relayModelProfile(connection, modelId)?.contextWindow; +} + /** * Mirrors @ai-sdk/openai@4.0.42 priority-processing detection. The UI and * runtime share this gate so a saved Fast declaration always reaches the wire. diff --git a/packages/core/src/session.ts b/packages/core/src/session.ts index 4d7c85ea0a..629b85a07e 100644 --- a/packages/core/src/session.ts +++ b/packages/core/src/session.ts @@ -792,6 +792,10 @@ export function userFacingText(message: Pick( ], [ 'policyName', - 'maxHistoryEstimatedTokens', 'prunedToolResults', 'prunedToolResultEstimatedTokensBefore', 'prunedToolResultEstimatedTokensAfter', @@ -105,6 +104,7 @@ const CURRENT_CONTEXT_BUDGET_SHAPE = defineObjectShape( * produce them through ContextBudgetDiagnostic. */ const RETIRED_CONTEXT_BUDGET_KEYS = [ + 'maxHistoryEstimatedTokens', 'maxHistoryTurns', 'semanticCompactEnabled', 'semanticCompactMode', @@ -285,38 +285,49 @@ export interface TokenUsageFields { providerRequestTraceId?: string; /** * The send's LAST provider request, as a pair: the input tokens the provider - * reported for it, and the wire payload chars the runtime measured for that - * same request. `input` above is the send's sum across steps and cannot - * anchor anything; this pair can, so the next turn estimates its first - * request from real usage instead of guessing the whole payload at char/4. - * - * Only the pair means anything — an anchor taken from one request and a - * baseline from another is off by a whole step's growth — so the two numbers - * live in one object that is written and read together. Absent means no - * anchor, and the estimate falls back to the cold start. + * reported for it, and its output tokens. `input` above is the send's sum + * across steps and cannot anchor anything; the last step's real input and + * output can, so the next turn judges its first request from real usage. + * Absent means no anchor, and the next turn has no proactive fold until its + * first accepted request. */ lastRequestAnchor?: LastRequestAnchor; } -/** Real input tokens of one provider request, paired with its measured payload chars. */ +/** + * The last provider request of a send, as the provider counted it: its real + * input tokens and its real output tokens. Together they are the baseline the + * next request is judged from — everything the model produced is re-sent as + * input — with no local measure involved (#4559). + * + * `payloadChars` is a retired key from the 0.2.0 anchor, which paired the input + * count with a locally measured payload size. It is still accepted on decode so + * sessions written by that build keep loading, and ignored. + */ export interface LastRequestAnchor { inputTokens: number; - payloadChars: number; + outputTokens?: number; } const LAST_REQUEST_ANCHOR_SHAPE = defineObjectShape()( - ['inputTokens', 'payloadChars'], - [], + ['inputTokens'], + ['outputTokens'], ); +const RETIRED_LAST_REQUEST_ANCHOR_KEYS = ['payloadChars'] as const; +const LAST_REQUEST_ANCHOR_DECODE_SHAPE = { + required: LAST_REQUEST_ANCHOR_SHAPE.required, + allowed: new Set([...LAST_REQUEST_ANCHOR_SHAPE.allowed, ...RETIRED_LAST_REQUEST_ANCHOR_KEYS]), +}; export function isLastRequestAnchor(value: unknown): value is LastRequestAnchor { return ( isRecord(value) && - hasExactShape(value, LAST_REQUEST_ANCHOR_SHAPE) && + hasExactShape(value, LAST_REQUEST_ANCHOR_DECODE_SHAPE) && isFiniteNumber(value.inputTokens) && - isFiniteNumber(value.payloadChars) && value.inputTokens > 0 && - value.payloadChars > 0 + (value.outputTokens === undefined || + (isFiniteNumber(value.outputTokens) && value.outputTokens >= 0)) && + (value.payloadChars === undefined || isFiniteNumber(value.payloadChars)) ); } diff --git a/packages/core/src/usage-stats/types.ts b/packages/core/src/usage-stats/types.ts index 53a6864eab..ae312683b5 100644 --- a/packages/core/src/usage-stats/types.ts +++ b/packages/core/src/usage-stats/types.ts @@ -284,7 +284,6 @@ export interface CompactionDecisionDiagnostic { export interface ContextBudgetDiagnostic { enabled: boolean; policyName?: string; - maxHistoryEstimatedTokens?: number; estimatedTokensBefore: number; estimatedTokensAfter: number; keptTurns: number; diff --git a/packages/runtime-host/src/__tests__/execution-model-composition.test.ts b/packages/runtime-host/src/__tests__/execution-model-composition.test.ts index 782695d39b..f63660ced5 100644 --- a/packages/runtime-host/src/__tests__/execution-model-composition.test.ts +++ b/packages/runtime-host/src/__tests__/execution-model-composition.test.ts @@ -1779,6 +1779,16 @@ test('production Host executes a canonical ai-sdk Session against a real provide }); assert.equal(configured.kind, 'committed'); await publishConnectionModel(policy, connection.connectionId, MODEL_ID); + // The fetched /models value is metadata only. This explicit model-facts + // declaration is the Maka compaction target used by the long-session flow. + await writeFile( + join(root, 'model-facts.json'), + JSON.stringify({ + schemaVersion: 1, + overrides: { [`moonshot:${MODEL_ID}`]: { contextWindow: 3_072 } }, + }), + 'utf8', + ); let policySnapshot = await policy.runtimePolicy.getSnapshot(); const personalized = await policy.runtimePolicy.mutate({ expectedRevision: policySnapshot.revision, @@ -1860,8 +1870,8 @@ test('production Host executes a canonical ai-sdk Session against a real provide assert.equal(remembered.result.kind, 'committed'); const turnIds: string[] = []; - // Cross the history high-water without making the text-only compact input - // exceed this fixture's 2,304-token summarizer budget. + // Cross the explicitly declared Maka window without making the text-only + // compact input exceed this fixture's 2,304-token summarizer budget. for (let index = 0; index < 5; index += 1) { const turnId = randomUUID(); turnIds.push(turnId); diff --git a/packages/runtime-host/src/protocol/index.ts b/packages/runtime-host/src/protocol/index.ts index 90e7197a60..645a01e056 100644 --- a/packages/runtime-host/src/protocol/index.ts +++ b/packages/runtime-host/src/protocol/index.ts @@ -100,7 +100,15 @@ export const RUNTIME_HOST_REGISTRATION_SCHEMA_VERSION = 1 as const; export const RUNTIME_HOST_PROTOCOL_VERSION = 0 as const; // Increment when the same protocol version no longer guarantees safe Client-Host // interoperability. Mismatches are rejected before domain commands are admitted. -export const RUNTIME_HOST_COMPATIBILITY_EPOCH = 104 as const; +export const RUNTIME_HOST_COMPATIBILITY_EPOCH = 105 as const; +// 105: Session transcripts gain four `system_note` kinds +// (`context_provider_dropping`, `context_window_suggestion`, +// `context_window_overrun`, `context_reported_window_exceeded`) and +// `token_usage` +// records reshape `lastRequestAnchor` to `{ inputTokens, outputTokens }`, all +// behind closed allowlists in @maka/core. An older client that handshakes +// would fail `decodeStoredMessage` on the first transcript carrying them, so +// the pair must refuse each other at the handshake instead (#4559). // 104: WorkHub Coordination actions add closed direct-stop proposals, // confirmations, expected-state preconditions, and outcomes. Older peers // reject these strict shapes. diff --git a/packages/runtime-host/src/server/execution-model-composition.ts b/packages/runtime-host/src/server/execution-model-composition.ts index e5dd3bd2e6..5688fb68c6 100644 --- a/packages/runtime-host/src/server/execution-model-composition.ts +++ b/packages/runtime-host/src/server/execution-model-composition.ts @@ -392,7 +392,7 @@ async function buildHostAiSdkBackend( : {}), ...(!input.context.tools && input.childAgents ? input.childAgents : {}), providerOptions, - contextBudget: buildDefaultContextBudgetPolicy(target.connection, { + contextBudget: buildDefaultContextBudgetPolicy({ name: 'runtime-host-default-history-budget', modelId: target.model, }), diff --git a/packages/runtime/src/__tests__/ai-sdk-backend.test.ts b/packages/runtime/src/__tests__/ai-sdk-backend.test.ts index a021546196..f9376cbee2 100644 --- a/packages/runtime/src/__tests__/ai-sdk-backend.test.ts +++ b/packages/runtime/src/__tests__/ai-sdk-backend.test.ts @@ -623,97 +623,6 @@ function memoryFinishTextChunks(delta: string): LanguageModelV4StreamPart[] { } describe('AiSdkBackend Memory Extraction triggers', () => { - test('dispatches a pre-turn Compaction recipe without projecting history or awaiting it', async () => { - const model = completionModel(); - const recorded: HistoryCompactCheckpoint[] = []; - let snapshot: MemoryExtractionSourceSnapshot | undefined; - let systemPromptResolutions = 0; - const backend = createTestAiSdkBackend({ - sessionId: 'session-1', - header: header(), - appendMessage: async () => {}, - connection: connection(), - apiKey: 'sk-test', - modelId: 'mock-model-id', - modelFactory: () => model, - systemPrompt: async () => { - systemPromptResolutions += 1; - return 'CURRENT_MEMORY_SYSTEM_PROMPT'; - }, - tools: [], - contextBudget: { - maxHistoryEstimatedTokens: 1_500, - charsPerToken: 1, - historyCompact: { - enabled: true, - }, - }, - summarizeHistoryCompact: async () => structuredSummary('AUTOMATIC_MEMORY_SUMMARY'), - recordHistoryCompactCheckpoint: (checkpoint) => { - recorded.push(checkpoint); - }, - memoryExtraction: { - gate: () => new Promise(() => {}), - automaticGate: () => ({ allowed: true }), - remember: async () => ({ status: 'unavailable', requestedItems: [] }), - extract: (value) => { - snapshot = value; - return new Promise(() => {}); - }, - }, - newId: idGenerator(), - now: monotonicClock(), - }); - const runtimeContext = [ - runtimeTextEvent({ - id: 'memory-compact-old-user', - turnId: 'memory-compact-turn-1', - role: 'user', - author: 'user', - text: 'The project uses SQLite. '.repeat(40), - }), - runtimeTextEvent({ - id: 'memory-compact-old-model', - turnId: 'memory-compact-turn-2', - role: 'model', - author: 'agent', - text: 'Acknowledged. '.repeat(70), - }), - runtimeTextEvent({ - id: 'memory-compact-boundary', - turnId: 'memory-compact-turn-3', - role: 'user', - author: 'user', - text: 'Keep this retained context.', - }), - ]; - - await drain( - backend.send({ - turnId: 'memory-compact-current', - runId: 'memory-compact-current-run', - text: 'continue', - context: [], - runtimeContext, - }), - ); - - assert.equal(recorded.length, 1); - assert.equal(recorded[0]?.memoryExtractionBoundary?.runtimeEventId, 'memory-compact-boundary'); - assert.equal(snapshot?.trigger, 'compaction'); - assert.equal(snapshot?.compactionCheckpointId, recorded[0]?.checkpointId); - assert.equal(snapshot?.compactionBoundaryEventId, 'memory-compact-boundary'); - assert.equal(snapshot?.sourceSystemPrompt, undefined); - assert.deepEqual(snapshot?.sourceMessages, []); - assert.deepEqual(snapshot?.sourceTools, {}); - assert.deepEqual(snapshot?.sourceActiveTools, []); - assert.equal(snapshot?.sourceProviderOptions, undefined); - assert.equal(snapshot?.rebuildSourceContextFromCompactionCheckpoint, true); - assert.equal(systemPromptResolutions, 1); - assert.match(JSON.stringify(model.doStreamCalls[0]), /CURRENT_MEMORY_SYSTEM_PROMPT/); - assert.equal(model.doStreamCalls.length, 1, 'the unresolved extraction must not block Agent'); - }); - test('terminates cleanly when the dynamic system prompt rejects before Compaction', async () => { const model = completionModel(); const recorded: HistoryCompactCheckpoint[] = []; @@ -731,7 +640,6 @@ describe('AiSdkBackend Memory Extraction triggers', () => { }, tools: [], contextBudget: { - maxHistoryEstimatedTokens: 1_500, charsPerToken: 1, historyCompact: { enabled: true }, }, @@ -775,154 +683,6 @@ describe('AiSdkBackend Memory Extraction triggers', () => { ); }); - for (const gate of [ - { allowed: false as const, reason: 'disabled' as const }, - { allowed: false as const, reason: 'incognito' as const }, - ]) { - test(`persists a denied marker and does not dispatch when automatic Compaction is ${gate.reason}`, async () => { - const recorded: HistoryCompactCheckpoint[] = []; - let dispatches = 0; - const backend = createTestAiSdkBackend({ - sessionId: 'session-1', - header: header(), - appendMessage: async () => {}, - connection: connection(), - apiKey: 'sk-test', - modelId: 'mock-model-id', - modelFactory: () => completionModel(), - tools: [], - contextBudget: { - maxHistoryEstimatedTokens: 1_500, - charsPerToken: 1, - historyCompact: { - enabled: true, - }, - }, - summarizeHistoryCompact: async () => structuredSummary('DENIED_MEMORY_SUMMARY'), - recordHistoryCompactCheckpoint: (checkpoint) => { - recorded.push(checkpoint); - }, - memoryExtraction: { - gate: async () => gate, - automaticGate: () => gate, - remember: async () => ({ status: 'unavailable', requestedItems: [] }), - extract: () => { - dispatches += 1; - }, - }, - newId: idGenerator(), - now: monotonicClock(), - }); - - await drain( - backend.send({ - turnId: `denied-${gate.reason}-current`, - runId: `denied-${gate.reason}-run`, - text: 'continue', - context: [], - runtimeContext: [ - runtimeTextEvent({ - id: `denied-${gate.reason}-old-user`, - turnId: 'denied-old-1', - role: 'user', - author: 'user', - text: 'Private disabled-period context. '.repeat(50), - }), - runtimeTextEvent({ - id: `denied-${gate.reason}-old-model`, - turnId: 'denied-old-2', - role: 'model', - author: 'agent', - text: 'Acknowledged. '.repeat(70), - }), - runtimeTextEvent({ - id: `denied-${gate.reason}-boundary`, - turnId: 'denied-old-3', - role: 'user', - author: 'user', - text: 'Retained tail.', - }), - ], - }), - ); - - assert.equal(recorded.length, 1); - assert.equal(recorded[0]?.memoryExtractionBoundary?.disposition, 'policy_denied'); - assert.equal(dispatches, 0); - }); - } - - test('keeps a transiently unavailable automatic Compaction checkpoint recoverable', async () => { - const recorded: HistoryCompactCheckpoint[] = []; - let dispatches = 0; - const backend = createTestAiSdkBackend({ - sessionId: 'session-1', - header: header(), - appendMessage: async () => {}, - connection: connection(), - apiKey: 'sk-test', - modelId: 'mock-model-id', - modelFactory: () => completionModel(), - tools: [], - contextBudget: { - maxHistoryEstimatedTokens: 1_500, - charsPerToken: 1, - historyCompact: { - enabled: true, - }, - }, - summarizeHistoryCompact: async () => structuredSummary('UNAVAILABLE_MEMORY_SUMMARY'), - recordHistoryCompactCheckpoint: (checkpoint) => { - recorded.push(checkpoint); - }, - memoryExtraction: { - gate: async () => ({ allowed: false, reason: 'unavailable' }), - automaticGate: () => ({ allowed: false, reason: 'unavailable' }), - remember: async () => ({ status: 'unavailable', requestedItems: [] }), - extract: () => { - dispatches += 1; - }, - }, - newId: idGenerator(), - now: monotonicClock(), - }); - - await drain( - backend.send({ - turnId: 'unavailable-current', - runId: 'unavailable-run', - text: 'continue', - context: [], - runtimeContext: [ - runtimeTextEvent({ - id: 'unavailable-old-user', - turnId: 'unavailable-old-1', - role: 'user', - author: 'user', - text: 'Recoverable context. '.repeat(50), - }), - runtimeTextEvent({ - id: 'unavailable-old-model', - turnId: 'unavailable-old-2', - role: 'model', - author: 'agent', - text: 'Acknowledged. '.repeat(70), - }), - runtimeTextEvent({ - id: 'unavailable-boundary', - turnId: 'unavailable-old-3', - role: 'user', - author: 'user', - text: 'Retained tail.', - }), - ], - }), - ); - - assert.equal(recorded[0]?.memoryExtractionBoundary?.disposition, 'eligible'); - assert.equal(dispatches, 0); - }); - test('exposes explicitly unsupported Memory triggers on the native OpenAI Responses lane', async () => { let modelCalls = 0; let memoryCalled = false; @@ -4634,7 +4394,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'manual-v2-compact-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, }, summarizeHistoryCompact: async () => structuredSummary('MANUAL_V2_HISTORY_COMPACT_SENTINEL'), @@ -4705,7 +4464,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'manual-single-turn-compact-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, }, summarizeHistoryCompact: async () => structuredSummary('MANUAL_SINGLE_TURN_SENTINEL'), @@ -4787,7 +4545,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'manual-v2-roll-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, }, loadHistoryCompactCheckpoint: () => previous, @@ -4875,7 +4632,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'manual-v2-reuse-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, }, loadHistoryCompactCheckpoint: () => previous, @@ -4902,203 +4658,6 @@ describe('AiSdkBackend model history', () => { assert.equal(result.contextBudget?.compactionDecisions?.[0]?.reason, 'already_compacted'); }); - test('manual compactHistory rewrites a fully covered checkpoint that exceeds current limits', async () => { - const oldEvents = [ - runtimeTextEvent({ - id: 'manual-v2-refit-old-1', - turnId: 'manual-v2-refit-turn-1', - role: 'user', - author: 'user', - text: 'manual v2 refit old alpha '.repeat(12), - }), - runtimeTextEvent({ - id: 'manual-v2-refit-old-2', - turnId: 'manual-v2-refit-turn-2', - role: 'model', - author: 'agent', - text: 'manual v2 refit old beta '.repeat(12), - }), - ]; - const previous = buildHistoryCompactCheckpoint({ - sessionId: 'session-1', - coveredRuntimeEvents: oldEvents, - summary: 'OVERSIZED_PREVIOUS_SUMMARY '.repeat(100), - summaryFormat: 'legacy_freeform', - charsPerToken: 1, - }); - - for (const limits of [ - { maxHistoryEstimatedTokens: 10_000 }, - { maxHistoryEstimatedTokens: 1_400 }, - ]) { - let summarizeCalls = 0; - const recorded: HistoryCompactCheckpoint[] = []; - const backend = createTestAiSdkBackend({ - sessionId: 'session-1', - header: header(), - appendMessage: async () => {}, - connection: connection(), - apiKey: 'sk-test', - modelId: 'mock-model-id', - modelFactory: () => completionModel(), - tools: [], - newId: idGenerator(), - now: monotonicClock(), - contextBudget: { - name: 'manual-v2-refit-test', - maxHistoryEstimatedTokens: limits.maxHistoryEstimatedTokens, - charsPerToken: 1, - historyCompact: { enabled: true }, - }, - loadHistoryCompactCheckpoint: () => previous, - summarizeHistoryCompact: async () => { - summarizeCalls += 1; - return structuredSummary('REFITTED_SUMMARY'); - }, - recordHistoryCompactCheckpoint: (checkpoint) => { - recorded.push(checkpoint); - }, - }); - - const result = await backend.compactHistory({ - turnId: 'turn-compact', - runId: 'run-1', - runtimeContext: [ - ...oldEvents, - runtimeTextEvent({ - id: 'manual-v2-refit-recent', - turnId: 'manual-v2-refit-recent-turn', - role: 'user', - author: 'user', - text: 'manual v2 refit retained context', - }), - ], - }); - - assert.equal(summarizeCalls, 1); - assert.equal(recorded.length, 1); - assert.equal(result.contextBudget?.compactionDecisions?.[0]?.decision, 'replaced'); - } - }); - - test('manual compactHistory does not record a rebuilt checkpoint whose envelope exceeds current limits', async () => { - let recordCalls = 0; - const backend = createTestAiSdkBackend({ - sessionId: 'session-1', - header: header(), - appendMessage: async () => {}, - connection: connection(), - apiKey: 'sk-test', - modelId: 'mock-model-id', - modelFactory: () => completionModel(), - tools: [], - newId: idGenerator(), - now: monotonicClock(), - contextBudget: { - name: 'manual-v2-envelope-budget-test', - maxHistoryEstimatedTokens: 100, - charsPerToken: 1, - historyCompact: { enabled: true }, - }, - summarizeHistoryCompact: async () => structuredSummary('TINY_SUMMARY'), - recordHistoryCompactCheckpoint: () => { - recordCalls += 1; - }, - }); - - const result = await backend.compactHistory({ - turnId: 'turn-compact', - runId: 'run-1', - runtimeContext: [ - runtimeTextEvent({ - id: 'manual-v2-envelope-old-1', - turnId: 'manual-v2-envelope-turn-1', - role: 'user', - author: 'user', - text: 'old alpha '.repeat(20), - }), - runtimeTextEvent({ - id: 'manual-v2-envelope-old-2', - turnId: 'manual-v2-envelope-turn-2', - role: 'model', - author: 'agent', - text: 'old beta '.repeat(20), - }), - runtimeTextEvent({ - id: 'manual-v2-envelope-recent', - turnId: 'manual-v2-envelope-recent-turn', - role: 'user', - author: 'user', - text: 'recent tail', - }), - ], - }); - - assert.equal(recordCalls, 0); - assert.deepEqual(result.outcome, { kind: 'failed', reason: 'prefix_over_budget' }); - }); - - test('manual compactHistory rejects a complete summary that makes the full replay larger', async () => { - let recordCalls = 0; - const backend = createTestAiSdkBackend({ - sessionId: 'session-1', - header: header(), - appendMessage: async () => {}, - connection: connection(), - apiKey: 'sk-test', - modelId: 'mock-model-id', - modelFactory: () => completionModel(), - tools: [], - newId: idGenerator(), - now: monotonicClock(), - contextBudget: { - name: 'manual-v2-larger-replacement-test', - maxHistoryEstimatedTokens: 10_000, - charsPerToken: 1, - historyCompact: { enabled: true }, - }, - summarizeHistoryCompact: async () => structuredSummary('LARGER_SUMMARY '.repeat(100)), - recordHistoryCompactCheckpoint: () => { - recordCalls += 1; - }, - }); - - const result = await backend.compactHistory({ - turnId: 'turn-compact', - runId: 'run-1', - runtimeContext: [ - runtimeTextEvent({ - id: 'manual-v2-larger-old-1', - turnId: 'manual-v2-larger-turn-1', - role: 'user', - author: 'user', - text: 'old alpha', - }), - runtimeTextEvent({ - id: 'manual-v2-larger-old-2', - turnId: 'manual-v2-larger-turn-2', - role: 'model', - author: 'agent', - text: 'old beta', - }), - runtimeTextEvent({ - id: 'manual-v2-larger-recent', - turnId: 'manual-v2-larger-recent-turn', - role: 'user', - author: 'user', - text: 'recent tail', - }), - ], - }); - - assert.equal(recordCalls, 0); - assert.deepEqual(result.outcome, { kind: 'failed', reason: 'replacement_not_smaller' }); - assert.equal( - result.contextBudget?.compactionDecisions?.[0]?.failOpenReason, - 'replacement_not_smaller', - ); - }); - test('manual compactHistory reports output-length exhaustion instead of empty_summary', async () => { const backend = createTestAiSdkBackend({ sessionId: 'session-1', @@ -5113,7 +4672,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'manual-v2-output-length-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, historyCompact: { enabled: true }, }, @@ -5167,7 +4725,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'manual-v2-write-gate-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, historyCompact: { enabled: true }, }, @@ -5225,7 +4782,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'malformed-summary-circuit-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, historyCompact: { enabled: true }, }, @@ -5329,7 +4885,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'malformed-summary-repair-circuit-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, historyCompact: { enabled: true }, }, @@ -5415,7 +4970,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'malformed-summary-cancel-circuit-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, historyCompact: { enabled: true }, }, @@ -5499,7 +5053,7 @@ describe('AiSdkBackend model history', () => { change: (input) => { input.contextBudget = { ...input.contextBudget, - maxHistoryEstimatedTokens: 12_000, + name: 'malformed-summary-config-circuit-changed', }; }, }, @@ -5529,7 +5083,6 @@ describe('AiSdkBackend model history', () => { readExecutionBoundary: readExternalExecutionBoundary, contextBudget: { name: 'malformed-summary-config-circuit-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, historyCompact: { enabled: true }, }, @@ -5632,7 +5185,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'manual-compact-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, }, }); @@ -5698,7 +5250,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'manual-compact-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, }, summarizeHistoryCompact: async () => structuredSummary('WRITE_FAILURE_SUMMARY'), @@ -5740,7 +5291,6 @@ describe('AiSdkBackend model history', () => { now: monotonicClock(), contextBudget: { name: 'manual-compact-abort-test', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, }, summarizeHistoryCompact: ({ abortSignal }) => @@ -6025,7 +5575,6 @@ describe('AiSdkBackend model history', () => { newId: idGenerator(), now: monotonicClock(), contextBudget: { - maxHistoryEstimatedTokens: 1_500, charsPerToken: 1, historyCompact: { enabled: true, @@ -6121,7 +5670,6 @@ describe('AiSdkBackend model history', () => { modelFactory: () => model, tools: [], contextBudget: { - maxHistoryEstimatedTokens: 100_000, historyCompact: { enabled: true }, }, loadHistoryCompactCheckpoint: () => checkpoint, @@ -6190,7 +5738,6 @@ describe('AiSdkBackend model history', () => { modelFactory: () => model, tools: [], contextBudget: { - maxHistoryEstimatedTokens: 100_000, historyCompact: { enabled: true }, }, loadHistoryCompactCheckpoint: () => checkpoint, @@ -6283,7 +5830,6 @@ describe('AiSdkBackend model history', () => { modelFactory: () => model, tools: [], contextBudget: { - maxHistoryEstimatedTokens: 100_000, historyCompact: { enabled: true }, }, loadHistoryCompactCheckpoint: () => checkpoint, @@ -6377,7 +5923,6 @@ describe('AiSdkBackend model history', () => { modelFactory: () => model, tools: [], contextBudget: { - maxHistoryEstimatedTokens: 100_000, historyCompact: { enabled: true }, }, loadHistoryCompactCheckpoint: () => checkpoint, @@ -9283,8 +8828,8 @@ describe('AiSdkBackend context budget and prompt attribution', () => { systemPrompt: 'durable system', contextBudget: { name: 'test-budget', - maxHistoryEstimatedTokens: 1_000, charsPerToken: 1, + historyCompact: { enabled: true }, }, }); diff --git a/packages/runtime/src/__tests__/context-budget-mid-turn-policy.test.ts b/packages/runtime/src/__tests__/context-budget-mid-turn-policy.test.ts index 6f36d1ea44..c8edb6bb95 100644 --- a/packages/runtime/src/__tests__/context-budget-mid-turn-policy.test.ts +++ b/packages/runtime/src/__tests__/context-budget-mid-turn-policy.test.ts @@ -22,6 +22,7 @@ import { describe, test } from 'node:test'; import type { LlmConnection } from '@maka/core/llm-connections'; import { buildDefaultContextBudgetPolicy, + resolveDeclaredContextWindow, resolveSelectedModelContextWindow, } from '../context-budget-policy.js'; @@ -46,9 +47,9 @@ test('context policy is independent of process environment overrides', () => { ); try { for (const name of Object.keys(overrides)) delete process.env[name]; - const baseline = buildDefaultContextBudgetPolicy(connection()); + const baseline = buildDefaultContextBudgetPolicy(); Object.assign(process.env, overrides); - assert.deepEqual(buildDefaultContextBudgetPolicy(connection()), baseline); + assert.deepEqual(buildDefaultContextBudgetPolicy(), baseline); } finally { for (const [name, value] of Object.entries(previous)) { if (value === undefined) delete process.env[name]; @@ -59,62 +60,15 @@ test('context policy is independent of process environment overrides', () => { describe('mid-turn history compact policy', () => { test('is owned by the runtime default', () => { - const policy = buildDefaultContextBudgetPolicy(connection()); + const policy = buildDefaultContextBudgetPolicy(); assert.equal(policy?.historyCompact?.enabled, true); - assert.deepEqual(policy?.historyCompact?.midTurn, { enabled: true, reserveTokens: 16_384 }); - }); -}); - -describe('window-bounded reserve derivation (issue #882 PR 3 review P2)', () => { - test('caps the derived reserve on a small-window model instead of degrading to a 1-token budget', () => { - // gpt-4 has an 8192-token window. A flat 16384 reserve used to derive - // maxHistoryEstimatedTokens = max(1, 8192 - 16384) = 1, and a mid_turn - // high water clamped to 1 token — every multi-step turn ran the - // summarizer for a checkpoint that could never pass the replay gate. - // The default reserve must be bounded by the KNOWN window: a quarter of - // the window, capped at the classic 16384. - const policy = buildDefaultContextBudgetPolicy(gpt4Connection(), { modelId: 'gpt-4' }); - assert.equal(policy?.maxHistoryEstimatedTokens, 8192 - 2048); - assert.deepEqual(policy?.historyCompact?.midTurn, { enabled: true, reserveTokens: 2048 }); - }); - - test('uses the official Agent Plan default-model window instead of the unknown-model fallback', () => { - const policy = buildDefaultContextBudgetPolicy(agentPlanConnection()); - assert.equal(policy?.maxHistoryEstimatedTokens, 256_000 - 16_384); - assert.deepEqual(policy?.historyCompact?.midTurn, { enabled: true, reserveTokens: 16_384 }); - }); - - test('keeps the classic 16384 reserve when the window is unknown (metadata-less model)', () => { - const policy = buildDefaultContextBudgetPolicy( - { - ...gpt4Connection(), - defaultModel: 'custom-model', - models: [{ id: 'custom-model' }], - } as LlmConnection, - { modelId: 'custom-model' }, - ); - // No window: the flat 32_000 fallback budget and the classic reserve. - assert.equal(policy?.maxHistoryEstimatedTokens, 32_000); - assert.deepEqual(policy?.historyCompact?.midTurn, { enabled: true, reserveTokens: 16_384 }); - // Both are policy choices about how much history to keep. Neither is a - // fact about the model, so nothing derives a context window from them. - assert.equal( - resolveSelectedModelContextWindow( - { - ...gpt4Connection(), - defaultModel: 'custom-model', - models: [{ id: 'custom-model' }], - } as LlmConnection, - 'custom-model', - ), - undefined, - ); + assert.deepEqual(policy?.historyCompact?.midTurn, { enabled: true }); }); }); describe('tool-result prune policy', () => { test('uses bounded runtime defaults', () => { - const policy = buildDefaultContextBudgetPolicy(connection()); + const policy = buildDefaultContextBudgetPolicy(); assert.deepEqual(policy?.activeToolResultPrune, { enabled: true, maxCurrentResultEstimatedTokens: 2_048, @@ -168,17 +122,12 @@ describe('declared relay context window', () => { models: [{ id: 'reasoner-32k', contextWindow: 8_192 }], relayModelProfiles: { 'reasoner-32k': { contextWindow: 131_072 } }, }; - const policy = buildDefaultContextBudgetPolicy(relay, { modelId: 'reasoner-32k' }); - // Declared 131_072 wins: reserve 131_072/4 caps at 16_384. The fetched - // 8_192 row would have yielded 8_192 − 2_048, and no declaration at all - // would have fallen to the 32_000 unknown-model default. - assert.equal(policy?.maxHistoryEstimatedTokens, 131_072 - 16_384); - // Clearing the declaration falls back to the fetched row's window. + assert.equal(resolveDeclaredContextWindow(relay, 'reasoner-32k'), 131_072); + assert.deepEqual(buildDefaultContextBudgetPolicy().historyCompact?.midTurn, { enabled: true }); + // Clearing the declaration does not turn the fetched row into a Maka + // window; it is provider metadata and remains display-only. const undeclared: LlmConnection = { ...relay, relayModelProfiles: undefined }; - const fallback = buildDefaultContextBudgetPolicy(undeclared, { - modelId: 'reasoner-32k', - }); - assert.equal(fallback?.maxHistoryEstimatedTokens, 8_192 - 2_048); + assert.equal(resolveDeclaredContextWindow(undeclared, 'reasoner-32k'), undefined); }); test('a declared context window holds on any provider', () => { @@ -199,26 +148,9 @@ describe('declared relay context window', () => { models: [{ id: 'reasoner-32k', contextWindow: 8_192 }], relayModelProfiles: { 'reasoner-32k': { contextWindow: 131_072 } }, }; - const policy = buildDefaultContextBudgetPolicy(other, { modelId: 'reasoner-32k' }); - assert.equal(policy?.maxHistoryEstimatedTokens, 131_072 - 16_384); - // Absent stays absent: an undeclared model still reads the stored row. + assert.equal(resolveDeclaredContextWindow(other, 'reasoner-32k'), 131_072); + // Absent stays absent: an undeclared model still has no Maka threshold. const undeclared: LlmConnection = { ...other, relayModelProfiles: undefined }; - assert.equal( - buildDefaultContextBudgetPolicy(undeclared, { modelId: 'reasoner-32k' }) - ?.maxHistoryEstimatedTokens, - 8_192 - 2_048, - ); + assert.equal(resolveDeclaredContextWindow(undeclared, 'reasoner-32k'), undefined); }); }); - -function agentPlanConnection(): LlmConnection { - return { - slug: 'volcengine-agent-plan', - name: 'Volcengine Agent Plan', - providerType: 'volcengine-agent-plan', - defaultModel: 'ark-code-latest', - enabled: true, - createdAt: 1, - updatedAt: 1, - }; -} diff --git a/packages/runtime/src/__tests__/context-budget-model-facts.test.ts b/packages/runtime/src/__tests__/context-budget-model-facts.test.ts index 46eeeb73f1..d7e4a256f7 100644 --- a/packages/runtime/src/__tests__/context-budget-model-facts.test.ts +++ b/packages/runtime/src/__tests__/context-budget-model-facts.test.ts @@ -21,6 +21,7 @@ import assert from 'node:assert/strict'; import test from 'node:test'; import { buildDefaultContextBudgetPolicy, + resolveDeclaredContextWindow, resolveSelectedModelContextWindow, } from '../context-budget-policy.js'; @@ -33,7 +34,8 @@ test('context budgeting prefers a model input limit over its context window', () }; assert.equal(resolveSelectedModelContextWindow(connection, undefined), 600); - assert.equal(buildDefaultContextBudgetPolicy(connection)?.maxHistoryEstimatedTokens, 450); + assert.equal(resolveDeclaredContextWindow(connection, undefined), undefined); + assert.deepEqual(buildDefaultContextBudgetPolicy().historyCompact?.midTurn, { enabled: true }); }); test('invalid zero input limits do not disable the context-window fallback', () => { @@ -76,4 +78,17 @@ test('a model-facts context window is the authoritative user declaration', () => }; assert.equal(resolveSelectedModelContextWindow(connection, undefined), 200_000); + assert.equal(resolveDeclaredContextWindow(connection, undefined), 200_000); +}); + +test('a reported model context window is metadata, not a Maka declaration', () => { + const connection = { + slug: 'openai', + providerType: 'openai' as const, + defaultModel: 'reported-model', + models: [{ id: 'reported-model', contextWindow: 100_000 }], + }; + + assert.equal(resolveSelectedModelContextWindow(connection, undefined), 100_000); + assert.equal(resolveDeclaredContextWindow(connection, undefined), undefined); }); diff --git a/packages/runtime/src/__tests__/context-budget.test.ts b/packages/runtime/src/__tests__/context-budget.test.ts index d5ad6e5b8c..b8e0fdf874 100644 --- a/packages/runtime/src/__tests__/context-budget.test.ts +++ b/packages/runtime/src/__tests__/context-budget.test.ts @@ -21,7 +21,11 @@ import assert from 'node:assert/strict'; import { createHash } from 'node:crypto'; import { test } from 'node:test'; import type { RuntimeEvent } from '@maka/core/runtime-event'; -import { applyRuntimeEventContextBudget } from '../context-budget.js'; +import { + applyRuntimeEventContextBudget, + shouldAppendContextCompactedNote, + shouldAppendContextCompactionFailedOpenNote, +} from '../context-budget.js'; import { estimateRuntimeEventsTokens } from '../model-history.js'; import { buildHistoryCompactCheckpoint } from '../history-compact-checkpoint.js'; @@ -37,7 +41,6 @@ test('estimates only model-visible provider context', () => { test('capacity policy keeps the canonical ledger until a checkpoint replaces it', () => { const events = [textEvent('user', 'large history '.repeat(100))]; const result = applyRuntimeEventContextBudget(events, { - maxHistoryEstimatedTokens: 1, historyCompact: { enabled: true }, }); assert.deepEqual(result?.events, events); @@ -109,3 +112,60 @@ function toolResultEvent(id: string, result: string): RuntimeEvent { content: { kind: 'function_response', id: 'tool-call', name: 'Bash', result }, }; } + +test('compaction notes fire for a fold made by the request hook, not only for a replay', () => { + // Since #4486 every new fold happens in the request-projection hook + // (`activeStep`); the turn that was compacted must show the note in that + // turn, not one turn later when the checkpoint is replayed (#4559). + const shell = { + enabled: true, + estimatedTokensBefore: 1, + estimatedTokensAfter: 1, + keptTurns: 1, + droppedTurns: 0, + keptEvents: 1, + droppedEvents: 0, + }; + const decision = (stage: 'priorReplay' | 'activeStep', outcome: 'replaced' | 'failedOpen') => ({ + ...shell, + compactionDecisions: [ + { + stage, + sourceKind: 'runtimeEvents' as const, + decision: outcome, + boundaryKind: 'historyCompact' as const, + }, + ], + }); + assert.equal(shouldAppendContextCompactedNote(decision('activeStep', 'replaced')), true); + assert.equal(shouldAppendContextCompactedNote(decision('priorReplay', 'replaced')), true); + assert.equal(shouldAppendContextCompactedNote(decision('activeStep', 'failedOpen')), false); + assert.equal( + shouldAppendContextCompactionFailedOpenNote(decision('activeStep', 'failedOpen')), + true, + ); + assert.equal( + shouldAppendContextCompactionFailedOpenNote(decision('priorReplay', 'failedOpen')), + true, + ); + assert.equal( + shouldAppendContextCompactionFailedOpenNote(decision('priorReplay', 'replaced')), + false, + ); + // A non-history boundary never speaks as a history compaction. + assert.equal( + shouldAppendContextCompactedNote({ + ...shell, + compactionDecisions: [ + { + stage: 'activeStep', + sourceKind: 'runtimeEvents', + decision: 'replaced', + boundaryKind: 'activeToolResultPrune', + }, + ], + } as never), + false, + ); + assert.equal(shouldAppendContextCompactedNote(undefined), false); +}); diff --git a/packages/runtime/src/__tests__/history-compact-checkpoint.test.ts b/packages/runtime/src/__tests__/history-compact-checkpoint.test.ts index 5aa6a5aafa..30fba8c08c 100644 --- a/packages/runtime/src/__tests__/history-compact-checkpoint.test.ts +++ b/packages/runtime/src/__tests__/history-compact-checkpoint.test.ts @@ -599,25 +599,6 @@ describe('history compact checkpoint', () => { ); }); - test('the builder re-runs the size floor over the covered span it is handed', () => { - // Every construction seam has the covered events in hand — including - // copy — so a structurally valid but undersized summary cannot be - // rebuilt over a large span and keep the marker. - const bigEvent: RuntimeEvent = { - ...textEvent(0), - content: { kind: 'text', text: `big ${'x'.repeat(60_000)}` }, - }; - assert.throws( - () => - buildHistoryCompactCheckpoint({ - sessionId: 'session-1', - coveredRuntimeEvents: [bigEvent], - summary: STRUCTURED_SUMMARY, - }), - /summary failed validation: malformed_summary_too_small_for_fold/, - ); - }); - test('shape validation fails closed on an unknown summary format marker', () => { const stamped = buildHistoryCompactCheckpoint({ sessionId: 'session-1', @@ -945,15 +926,10 @@ describe('history compact checkpoint', () => { summaryFormat: 'legacy_freeform', }); - const replay = applyRuntimeEventHistoryCompact( - events, - { - enabled: true, - checkpoint, - }, - 1, - 1_000, - ); + const replay = applyRuntimeEventHistoryCompact(events, { + enabled: true, + checkpoint, + }); assert.equal(replay.events[0]?.id, `history-compact:${checkpoint.checkpointId}`); assert.match( @@ -967,7 +943,7 @@ describe('history compact checkpoint', () => { assert.equal(replay.checkpoint?.checkpointId, checkpoint.checkpointId); }); - test('replays a durable pre_turn checkpoint below the current high water', () => { + test('replays a durable pre_turn checkpoint without a local size gate', () => { const events = Array.from({ length: 6 }, (_, index) => ({ ...textEvent(index), content: { @@ -982,16 +958,11 @@ describe('history compact checkpoint', () => { summaryFormat: 'legacy_freeform', }); - // The raw history is deliberately far below high water. Once a durable + // The raw history is deliberately small. Once a durable // checkpoint exists, replaying it is nevertheless mandatory: otherwise a // recovery/manual compaction only affects its own turn and the next turn // resurrects the covered raw prefix. - const replay = applyRuntimeEventHistoryCompact( - events, - { enabled: true, checkpoint }, - 4, - 1_000_000, - ); + const replay = applyRuntimeEventHistoryCompact(events, { enabled: true, checkpoint }); assert.equal(replay.checkpoint?.checkpointId, checkpoint.checkpointId); assert.deepEqual( @@ -1000,72 +971,6 @@ describe('history compact checkpoint', () => { ); assert.equal(replay.diagnosticPatch.compactionDecisions?.[0]?.decision, 'replaced'); }); - - test('accepts a complete checkpoint above legacy block limits when the full replay fits', () => { - const events = Array.from({ length: 8 }, (_, index) => ({ - ...textEvent(index), - content: { - kind: 'text' as const, - text: `source-payload-${index} `.repeat(index < 4 ? 80 : 1), - }, - })); - const checkpoint = buildHistoryCompactCheckpoint({ - sessionId: 'session-1', - coveredRuntimeEvents: events.slice(0, 4), - summary: 'checkpoint summary '.repeat(20), - summaryFormat: 'legacy_freeform', - charsPerToken: 1, - }); - assert.ok(checkpoint.estimatedTokens > 100); - - const replay = applyRuntimeEventHistoryCompact( - events, - { - enabled: true, - checkpoint, - }, - 1, - 10_000, - ); - - assert.equal(replay.checkpoint?.checkpointId, checkpoint.checkpointId); - assert.equal( - replay.events.some((event) => event.id === `history-compact:${checkpoint.checkpointId}`), - true, - ); - }); - - test('applies max-history overrides to checkpoint replay validation', () => { - const events = Array.from({ length: 8 }, (_, index) => ({ - ...textEvent(index), - content: { kind: 'text' as const, text: `payload-${index} `.repeat(20) }, - })); - const checkpoint = buildHistoryCompactCheckpoint({ - sessionId: 'session-1', - coveredRuntimeEvents: events.slice(0, 6), - summary: 'short checkpoint', - summaryFormat: 'legacy_freeform', - charsPerToken: 1, - }); - const checkpointTokens = estimateRuntimeEventsTokens( - [historyCompactCheckpointToRuntimeEvent(checkpoint)], - 1, - ); - const overrideMax = checkpointTokens + 1; - - const replay = applyRuntimeEventHistoryCompact( - events, - { - enabled: true, - checkpoint, - }, - 1, - 10_000, - { maxHistoryEstimatedTokens: overrideMax }, - ); - - assert.equal(replay.checkpoint, undefined); - }); }); function textEvent(index: number): RuntimeEvent { diff --git a/packages/runtime/src/__tests__/history-compact-mid-turn-checkpoint.test.ts b/packages/runtime/src/__tests__/history-compact-mid-turn-checkpoint.test.ts index bfcae155d2..99b04fad39 100644 --- a/packages/runtime/src/__tests__/history-compact-mid-turn-checkpoint.test.ts +++ b/packages/runtime/src/__tests__/history-compact-mid-turn-checkpoint.test.ts @@ -298,12 +298,7 @@ describe('mid-turn history compact checkpoint', () => { // Normal thresholds: the raw projection is far below the default high // water, and the accepted mid_turn checkpoint must STILL replay — the // covered raw span may never be re-injected on recovery. - const replay = applyRuntimeEventHistoryCompact( - events, - { enabled: true, checkpoint }, - 1, - 10_000, - ); + const replay = applyRuntimeEventHistoryCompact(events, { enabled: true, checkpoint }); assert.equal(replay.checkpoint?.checkpointId, checkpoint.checkpointId); assert.deepEqual( diff --git a/packages/runtime/src/__tests__/history-compact-summarizer.test.ts b/packages/runtime/src/__tests__/history-compact-summarizer.test.ts index 2759154349..16cbbd3698 100644 --- a/packages/runtime/src/__tests__/history-compact-summarizer.test.ts +++ b/packages/runtime/src/__tests__/history-compact-summarizer.test.ts @@ -81,7 +81,7 @@ function inputWith(events: RuntimeEvent[], abortSignal?: AbortSignal): HistoryCo } describe('buildLlmHistorySummarizer', () => { - test('inherits the session provider options without imposing a compaction-only output cap', async () => { + test('inherits the session provider options and applies the default output cap', async () => { let seen: Parameters[0] | undefined; const providerOptions = { openaiCompatible: { reasoningEffort: 'high' } }; const summarize = buildLlmHistorySummarizer({ @@ -95,11 +95,10 @@ describe('buildLlmHistorySummarizer', () => { await summarize({ ...inputWith([ev({ role: 'user', author: 'user', content: { kind: 'text', text: 'hi' } })]), - inputBudget: { maxEstimatedTokens: 10_000, charsPerToken: 1 }, }); assert.strictEqual(seen?.providerOptions, providerOptions); - assert.strictEqual(seen?.maxOutputTokens, undefined); + assert.strictEqual(seen?.maxOutputTokens, 8_000); }); test('attributes provider-reported usage to one canonical history-compaction record', async () => { @@ -313,11 +312,14 @@ describe('buildLlmHistorySummarizer', () => { ['fc1', 'fc2'], ); const shape = messages.map((m) => `${m.role}:${m.content.map((part) => part.type).join('+')}`); - assert.deepEqual(shape.slice(-4), [ + // The request always ends on the summary instruction; the folded span is + // everything before it. + assert.deepEqual(shape.slice(-5), [ 'assistant:tool-call+tool-call', 'tool:tool-result', 'tool:tool-result', 'assistant:text', + 'user:text', ]); }); @@ -379,17 +381,48 @@ describe('buildLlmHistorySummarizer', () => { 'tool:tool-result', 'tool:tool-result', 'tool:tool-result', + 'user:text', ]); assert.deepEqual( messages[1]!.content.map((part) => part.toolCallId), ['fc1', 'fc2', 'fc3'], ); assert.deepEqual( - messages.slice(2).map((m) => m.content[0]!.toolCallId), + messages.slice(2, -1).map((m) => m.content[0]!.toolCallId), ['fc1', 'fc2', 'fc3'], ); }); + test('ends every summary request with a user instruction the model can answer', async () => { + // A chat-template model handed a conversation that ends on its own turn + // emits end-of-sequence and nothing else (observed on Ollama qwen2.5: + // finish `stop`, one output token, empty text). The request therefore + // closes with an instruction, on the first attempt and on the repair. + const seen: Array<{ messages: unknown[] }> = []; + const generateText: AiSdkGenerateTextLike = async (opts) => { + seen.push(opts); + return seen.length === 1 + ? { text: 'not a summary', finishReason: 'stop' } + : { text: VALID_SUMMARY, finishReason: 'stop' }; + }; + const summarize = buildLlmHistorySummarizer({ resolveModel: () => ({}), generateText }); + await summarize( + inputWith([ + ev({ role: 'user', author: 'user', content: { kind: 'text', text: 'question' } }), + ev({ role: 'model', author: 'agent', content: { kind: 'text', text: 'answer' } }), + ]), + ); + assert.equal(seen.length, 2); + for (const call of seen) { + const last = call.messages.at(-1) as { + role: string; + content: Array<{ type: string; text?: string }>; + }; + assert.equal(last.role, 'user'); + assert.match(last.content[0]!.text ?? '', /write the structured summary/i); + } + }); + test('does not merge distinct settled steps into one assistant message', async () => { const seen: Array<{ messages: unknown[] }> = []; const generateText: AiSdkGenerateTextLike = async (opts) => { @@ -435,146 +468,10 @@ describe('buildLlmHistorySummarizer', () => { 'tool:tool-result', 'assistant:tool-call', 'tool:tool-result', + 'user:text', ]); }); - test('bounds the oldest oversized tool result before dispatch while preserving newer context', async () => { - let seen: Parameters[0] | undefined; - const generateText: AiSdkGenerateTextLike = async (options) => { - seen = options; - // Proportionate to the large folded span so the size floor passes. - return { text: VALID_SUMMARY.replace('- done', `- ${'done '.repeat(200)}`) }; - }; - const summarize = buildLlmHistorySummarizer({ resolveModel: () => 'fake-model', generateText }); - const oldToolOutput = 'OLD_OVERSIZED_TOOL_OUTPUT_'.repeat(1_024); - const events: RuntimeEvent[] = [ - ev({ - role: 'model', - author: 'agent', - content: { kind: 'function_call', id: 'old-call', name: 'read', args: { path: 'old.log' } }, - }), - ev({ - role: 'tool', - author: 'tool', - content: { - kind: 'function_response', - id: 'old-call', - name: 'read', - result: oldToolOutput, - }, - }), - ev({ - role: 'model', - author: 'agent', - content: { - kind: 'function_call', - id: 'recent-call', - name: 'read', - args: { path: 'recent.log' }, - }, - }), - ev({ - role: 'tool', - author: 'tool', - content: { - kind: 'function_response', - id: 'recent-call', - name: 'read', - result: 'RECENT_TOOL_RESULT', - }, - }), - ev({ - role: 'model', - author: 'agent', - content: { kind: 'text', text: 'LATEST_GROUNDED_CONTEXT' }, - }), - ]; - - await summarize({ - ...inputWith(events), - inputBudget: { maxEstimatedTokens: 4_000, charsPerToken: 1 }, - }); - - const messages = seen!.messages; - const serialized = JSON.stringify(messages); - assert.ok(serialized.length <= 4_000); - assert.equal(serialized.includes(oldToolOutput), false); - assert.match(serialized, /Tool output omitted/); - assert.match(serialized, /RECENT_TOOL_RESULT/); - assert.match(serialized, /LATEST_GROUNDED_CONTEXT/); - assert.match(JSON.stringify(events), /OLD_OVERSIZED_TOOL_OUTPUT/); - assert.deepEqual( - messages.flatMap((message) => - typeof message.content === 'string' - ? [] - : message.content - .filter((part) => part.type === 'tool-call' || part.type === 'tool-result') - .map((part) => ({ type: part.type, toolCallId: part.toolCallId })), - ), - [ - { type: 'tool-call', toolCallId: 'old-call' }, - { type: 'tool-result', toolCallId: 'old-call' }, - { type: 'tool-call', toolCallId: 'recent-call' }, - { type: 'tool-result', toolCallId: 'recent-call' }, - ], - ); - }); - - test('fails with input_too_large before dispatch when non-tool history cannot fit', async () => { - let calls = 0; - let modelResolutions = 0; - const summarize = buildLlmHistorySummarizer({ - resolveModel: () => { - modelResolutions += 1; - return 'fake-model'; - }, - generateText: async () => { - calls += 1; - return { text: 'should not dispatch' }; - }, - }); - - await assert.rejects( - summarize({ - ...inputWith([ - ev({ - role: 'user', - author: 'user', - content: { kind: 'text', text: 'x'.repeat(1_000) }, - }), - ]), - inputBudget: { maxEstimatedTokens: 10, charsPerToken: 1 }, - }), - (error) => - error instanceof HistoryCompactSummarizerError && error.reason === 'input_too_large', - ); - assert.equal(calls, 0); - assert.equal(modelResolutions, 0); - }); - - test('charges the summarization instructions against the input budget before dispatch', async () => { - let calls = 0; - const summarize = buildLlmHistorySummarizer({ - resolveModel: () => 'fake-model', - generateText: async () => { - calls += 1; - return { text: 'should not dispatch' }; - }, - }); - - await assert.rejects( - summarize({ - ...inputWith([ - ev({ role: 'user', author: 'user', content: { kind: 'text', text: 'small history' } }), - ]), - inputBudget: { maxEstimatedTokens: 500, charsPerToken: 1 }, - }), - (error) => - error instanceof HistoryCompactSummarizerError && error.reason === 'input_too_large', - ); - assert.equal(calls, 0); - }); - test('stamped step ids decide membership over settledness', async () => { const seen: Array<{ messages: unknown[] }> = []; const generateText: AiSdkGenerateTextLike = async (opts) => { @@ -634,6 +531,7 @@ describe('buildLlmHistorySummarizer', () => { 'tool:tool-result', 'assistant:tool-call', 'tool:tool-result', + 'user:text', ]); assert.deepEqual( messages[0]!.content.map((part) => part.toolCallId), @@ -683,6 +581,29 @@ describe('buildLlmHistorySummarizer', () => { ); }); + test('shortens the prompt once when the first summary hits the output limit', async () => { + const instructions: string[] = []; + const summarize = buildLlmHistorySummarizer({ + resolveModel: () => 'fake-model', + generateText: async (options) => { + instructions.push(options.instructions); + return { + text: VALID_SUMMARY, + finishReason: instructions.length === 1 ? 'length' : 'stop', + }; + }, + }); + + assert.equal( + await summarize( + inputWith([ev({ role: 'user', author: 'user', content: { kind: 'text', text: 'hi' } })]), + ), + VALID_SUMMARY, + ); + assert.equal(instructions.length, 2); + assert.match(instructions[1] ?? '', /cut off at the output limit/); + }); + test('rejects the incident fragment: a free-form summary without the mandated sections', async () => { // The #3029 incident: 742 folded events accepted a 138-token free-form // fragment as their checkpoint. Section-less prose must fail open. @@ -771,6 +692,32 @@ describe('buildLlmHistorySummarizer', () => { assert.equal(calls, 2); }); + test('a context-length rejection of the repair request stays the input_too_large signal', async () => { + // The initial request fit; the stricter repair prompt is longer and the + // provider rejected it. That rejection must reach the planner as + // `input_too_large` so it retreats, not be filed under the initial defect. + let calls = 0; + const summarize = buildLlmHistorySummarizer({ + resolveModel: () => 'fake-model', + generateText: async () => { + calls += 1; + if (calls === 1) return { text: 'free-form incomplete summary', finishReason: 'stop' }; + throw new Error( + "This model's maximum context length is 1000 tokens. However, your messages exceed the context window.", + ); + }, + }); + + await assert.rejects( + summarize( + inputWith([ev({ role: 'user', author: 'user', content: { kind: 'text', text: 'hi' } })]), + ), + (error) => + error instanceof HistoryCompactSummarizerError && error.reason === 'input_too_large', + ); + assert.equal(calls, 2); + }); + test('preserves cancellation when a malformed-summary repair is aborted', async () => { let calls = 0; const abortError = Object.assign(new Error('stopped during repair'), { name: 'AbortError' }); @@ -1223,11 +1170,14 @@ describe('buildLlmHistorySummarizer', () => { assert.strictEqual(result, withInlineFence); }); - test('rejects a paragraph-sized summary for a large folded span', async () => { + test('rejects a paragraph-sized summary for a large folded span when usage says it is too small', async () => { const summarize = buildLlmHistorySummarizer({ resolveModel: () => 'fake-model', // Structurally complete, but far below the floor for a large fold. - generateText: async () => ({ text: VALID_SUMMARY }), + generateText: async () => ({ + text: VALID_SUMMARY, + usage: { inputTokens: 20_000, outputTokens: 50 }, + }), }); await assert.rejects( @@ -1244,9 +1194,43 @@ describe('buildLlmHistorySummarizer', () => { ); }); - test('the size floor covers the full replaced span, not just the newly folded increment', async () => { - // Steady-state roll-forward: the checkpoint replaces everything it covers, - // so a small increment must not let a fragment replace a large span. + test("a provider's context-length rejection is the fold's input_too_large signal", async () => { + // The summarizer's provider is the one judge of whether the fold fits its + // window. A fake provider that rejects above N characters must surface as + // `input_too_large`, the reason the planner retreats on, not as a generic + // provider error that fails the fold open. + const generateText: AiSdkGenerateTextLike = async (opts) => { + if (JSON.stringify(opts.messages).length > 2_000) { + throw new Error( + "This model's maximum context length is 1000 tokens. However, your messages exceed the context window.", + ); + } + return { text: VALID_SUMMARY, finishReason: 'stop' }; + }; + const summarize = buildLlmHistorySummarizer({ resolveModel: () => 'fake-model', generateText }); + await assert.rejects( + summarize( + inputWith([ + ev({ role: 'user', author: 'user', content: { kind: 'text', text: 'x'.repeat(5_000) } }), + ]), + ), + (error: unknown) => + error instanceof HistoryCompactSummarizerError && error.reason === 'input_too_large', + ); + assert.equal( + await summarize( + inputWith([ev({ role: 'user', author: 'user', content: { kind: 'text', text: 'small' } })]), + ), + VALID_SUMMARY, + ); + }); + + test('the usage floor judges an initial fold and stands down on a roll-forward', async () => { + // On an initial fold the summarizer's input IS the covered span, so a + // 50-token summary of a 20,000-token span is a fragment. On a roll-forward + // the input is the previous summary plus the increment, not the span, so + // the same numbers say nothing about the whole and the floor must not + // reject the fold on them. const old = ev({ role: 'user', author: 'user', @@ -1265,23 +1249,27 @@ describe('buildLlmHistorySummarizer', () => { }); const summarize = buildLlmHistorySummarizer({ resolveModel: () => 'fake-model', - generateText: async () => ({ text: VALID_SUMMARY }), + generateText: async () => ({ + text: VALID_SUMMARY, + usage: { inputTokens: 20_000, outputTokens: 50 }, + }), }); await assert.rejects( - summarize({ + summarize(inputWith([old, newer])), + /malformed_summary_too_small_for_fold/, + ); + assert.equal( + await summarize({ ...inputWith([old, newer]), previousCheckpoint, newlyFoldedRuntimeEvents: [newer], }), - /malformed_summary_too_small_for_fold/, + VALID_SUMMARY, ); }); - test('a summary at exactly the floor is accepted under ceil-based token estimates', async () => { - // 799 chars at 4 chars/token is ceil(799/4) = 200 estimated tokens — - // exactly the documented floor, so it must pass, not be rejected by a - // raw-character comparison. + test('a summary without provider usage is not rejected by the size floor', async () => { const skeleton = (progress: string) => `## Goal\nX\n\n## Progress\n- ${progress}\n\n## Next Steps\n1. continue\n\n## Critical Context\n- (none)`; const exactFloor = skeleton('p'.repeat(799 - skeleton('').length)); @@ -1379,7 +1367,6 @@ describe('buildLlmHistorySummarizer', () => { ...input, previousCheckpoint, newlyFoldedRuntimeEvents: [newer], - inputBudget: { maxEstimatedTokens: 10_000, charsPerToken: 1 }, }); assert.strictEqual(result, VALID_SUMMARY); diff --git a/packages/runtime/src/__tests__/history-compaction.test.ts b/packages/runtime/src/__tests__/history-compaction.test.ts index 9f941bf193..7dd3f7e580 100644 --- a/packages/runtime/src/__tests__/history-compaction.test.ts +++ b/packages/runtime/src/__tests__/history-compaction.test.ts @@ -21,8 +21,6 @@ import assert from 'node:assert/strict'; import { describe, test } from 'node:test'; import type { RuntimeEvent } from '@maka/core/runtime-event'; import { - estimateNextRequestTokens, - exceedsHighWater, applyRuntimeEventHistoryCompact, planHistoryCompaction, selectSafeCompactionPrefix, @@ -31,43 +29,6 @@ import { import { HistoryCompactSummarizerError } from '../history-compact-summarizer.js'; import { matchHistoryCompactCheckpointPrefix } from '../history-compact-checkpoint.js'; -describe('context compaction trigger measurement', () => { - test('anchors on real provider usage plus a tail char/4 delta', () => { - // last step: 100 input + 40 output real tokens, then 400 chars of new tool results - assert.equal( - estimateNextRequestTokens({ priorUsageTokens: 140, appendedChars: 400, charsPerToken: 4 }), - 140 + 100, - ); - }); - - test('credits a SIGNED negative payload delta after a compaction shrank the projection', () => { - // The last usage sample measured the PRE-compaction request; the payload - // delta is negative after the fold, so the estimate must shrink with it — - // clamping the delta at zero would judge the compacted request by the - // pre-compaction usage and wrongly exhaust a rescued turn. - assert.equal( - estimateNextRequestTokens({ priorUsageTokens: 700, appendedChars: -1_200, charsPerToken: 4 }), - 400, - ); - // The estimate never goes below zero even when the shrink exceeds usage. - assert.equal( - estimateNextRequestTokens({ priorUsageTokens: 100, appendedChars: -4_000, charsPerToken: 4 }), - 0, - ); - }); - - test('falls back to whole-projection char/4 on cold start (no usage)', () => { - // Unanchored, the caller passes the whole payload as the delta against a - // zero baseline, so the same formula yields the cold-start estimate. - assert.equal(estimateNextRequestTokens({ appendedChars: 800, charsPerToken: 4 }), 200); - }); - - test('high-water crosses at contextWindow minus reserve', () => { - assert.equal(exceedsHighWater(100_000, 128_000, 16_384), false); - assert.equal(exceedsHighWater(120_000, 128_000, 16_384), true); - }); -}); - describe('safe compaction prefix selection', () => { test('folds the largest immutable non-partial prefix, leaving the reserved tail', () => { const events = [ @@ -215,7 +176,7 @@ describe('plan context compaction', () => { assert.deepEqual(result.replacementEvents[1], events[2]); }); - test('backs the safe prefix down locally when the full summary input does not fit', async () => { + test('retreats the safe prefix by half for each input-too-large rejection', async () => { const events = [ user('old-user', 'old-turn'), model('old-model', 'old-turn', 'old result'), @@ -230,7 +191,7 @@ describe('plan context compaction', () => { reserveTailEvents: 0, summarize: ({ coveredRuntimeEvents }) => { attemptedCoverage.push(coveredRuntimeEvents.map((event) => event.id)); - if (coveredRuntimeEvents.length === events.length) { + if (attemptedCoverage.length <= 2) { throw new HistoryCompactSummarizerError('input_too_large'); } return structuredSummary('A bounded automatic summary.'); @@ -242,14 +203,26 @@ describe('plan context compaction', () => { if (result.decision !== 'compacted') return; assert.deepEqual(attemptedCoverage, [ ['old-user', 'old-model', 'recent-user', 'recent-model'], - ['old-user', 'old-model', 'recent-user'], + ['old-user', 'old-model'], + ['old-user'], ]); assert.deepEqual( result.tailRuntimeEvents.map((event) => event.id), - ['recent-model'], + ['old-model', 'recent-user', 'recent-model'], ); }); + test('fails open after repeated input-too-large retreat reaches no safe span', async () => { + const result = await planHistoryCompaction( + planInput({ + summarize: () => { + throw new HistoryCompactSummarizerError('input_too_large'); + }, + }), + ); + assert.deepEqual(result, { decision: 'fail_open', reason: 'no_safe_completed_span' }); + }); + test('persisted checkpoint replay-validates against the same ledger prefix (recovery)', async () => { const events = longTurnEvents(); const result = await planHistoryCompaction(planInput({ orderedEvents: events })); @@ -263,14 +236,12 @@ describe('plan context compaction', () => { assert.equal(match.coveredEventCount, result.coveredRuntimeEvents.length); // Normal thresholds: even though the raw ledger is far below the default - // high water, the accepted mid_turn checkpoint replays — recovery never + // local threshold, the accepted mid_turn checkpoint replays — recovery never // re-injects the replaced raw span. - const replay = applyRuntimeEventHistoryCompact( - events, - { enabled: true, checkpoint: result.checkpoint }, - 4, - 1_000_000, - ); + const replay = applyRuntimeEventHistoryCompact(events, { + enabled: true, + checkpoint: result.checkpoint, + }); assert.equal(replay.checkpoint?.checkpointId, result.checkpoint.checkpointId); const replayIds = replay.events.map((event) => event.id); assert.equal(replayIds[0], `history-compact:${result.checkpoint.checkpointId}`); diff --git a/packages/runtime/src/__tests__/mid-turn-capacity-backend.test.ts b/packages/runtime/src/__tests__/mid-turn-capacity-backend.test.ts index c528f6ae66..a98fa11b21 100644 --- a/packages/runtime/src/__tests__/mid-turn-capacity-backend.test.ts +++ b/packages/runtime/src/__tests__/mid-turn-capacity-backend.test.ts @@ -36,7 +36,6 @@ import { } from '../session-event-runtime-mapper.js'; import type { RuntimeEventMapContext } from '../session-event-runtime-mapper.js'; import { applyRuntimeEventContextBudget } from '../context-budget.js'; -import { evaluateHistoryCompactCheckpointReplay } from '../history-compaction.js'; import type { HistoryCompactCheckpoint, HistoryCompactProviderState, @@ -104,12 +103,15 @@ interface MidTurnFixtureOptions { contextWindow?: number; /** Omit the model's context window entirely (unknown model metadata). */ withoutContextWindow?: boolean; + /** Keep the model-reported window as metadata without making it a Maka target. */ + declareContextWindow?: boolean; + /** The model's declared output limit, a provider fact the trigger reserves for the reply. */ + modelMaxOutputTokens?: number; /** * Derive the policy from the runtime default (buildDefaultContextBudgetPolicy) * instead of the hand-built one, so a test can exercise the shipped default. */ useRuntimeDefaultPolicy?: boolean; - reserveTokens?: number; summarize?: ( input: HistoryCompactSummaryInput, ) => @@ -153,6 +155,8 @@ interface MidTurnFixtureOptions { assistantTextInFirstStep?: boolean; /** Override the first step's reported usage; 'missing' = empty usage object. */ firstStepUsage?: { input: number; output: number } | 'missing'; + /** Make the first provider step end at its output limit. */ + firstStepFinishReason?: 'length'; /** Override the final (text) step's reported usage. */ finalStepUsage?: { input: number; output: number }; /** Prior-turn RuntimeEvents appended after the shaped priors (e.g. a persisted usage anchor). */ @@ -161,8 +165,6 @@ interface MidTurnFixtureOptions { priorRunHeaders?: readonly AgentRunHeader[]; /** System prompt size sent through the provider's separate system field. */ systemPromptChars?: number; - /** Lower the pre-turn history-shaping threshold so that gate can be exercised. */ - maxHistoryEstimatedTokens?: number; /** An always-active tool whose schema dominates the request payload. */ bigActiveTool?: boolean; /** @@ -175,7 +177,10 @@ interface MidTurnFixtureOptions { captureMemoryExtraction?: boolean; memoryGate?: | { readonly allowed: true } - | { readonly allowed: false; readonly reason: 'disabled' | 'incognito' | 'unavailable' }; + | { + readonly allowed: false; + readonly reason: 'disabled' | 'incognito' | 'unavailable'; + }; } /** @@ -188,8 +193,13 @@ interface MidTurnFixtureOptions { type ConsumerMode = 'immediate' | 'slow'; function buildFixture(options: MidTurnFixtureOptions = {}): MidTurnFixture { - const contextWindow = options.contextWindow ?? 2_000; - const reserveTokens = options.reserveTokens ?? 1_500; + // Most cases model a user-declared target. Keep it between the first and + // second mock request baselines so the normal three-step journey folds once. + // Steps report 100/20, then 150/30, then 120/10, so the baselines are 120, + // 180 and 130 and the reserve (twice the last reply) is 40, 60 and 20. A + // default window of 190 keeps the first request inside it and crosses on the + // second, which is the journey these fixtures describe. + const contextWindow = options.contextWindow ?? 190; const recorded: HistoryCompactCheckpoint[] = []; const toolExecutions: string[] = []; const events: SessionEvent[] = []; @@ -211,9 +221,8 @@ function buildFixture(options: MidTurnFixtureOptions = {}): MidTurnFixture { outputTokens: { total: output, text: output, reasoning: 0 }, }); const firstStepUsage = (): ReturnType => { - // A usage object the SDK accepts but whose token counts are absent — the - // adapter's normalization fails closed (undefined) and the capacity hook's - // usability check must fall back to cold start (the finding-1 shape). + // A usage object the SDK accepts but whose token counts are absent. The + // adapter fails closed (undefined), so the capacity hook has no baseline. if (options.firstStepUsage === 'missing') return { inputTokens: {}, outputTokens: {} } as ReturnType; if (options.firstStepUsage) @@ -222,10 +231,18 @@ function buildFixture(options: MidTurnFixtureOptions = {}): MidTurnFixture { }; const toolCallChunks = (id: string, name: string, args: object): LanguageModelV4StreamPart[] => [ { type: 'stream-start', warnings: [] }, - { type: 'tool-call', toolCallId: id, toolName: name, input: JSON.stringify(args) }, + { + type: 'tool-call', + toolCallId: id, + toolName: name, + input: JSON.stringify(args), + }, { type: 'finish', - finishReason: { unified: 'tool-calls', raw: 'tool_calls' }, + finishReason: + id === 'tool-1' && options.firstStepFinishReason === 'length' + ? { unified: 'length', raw: 'length' } + : { unified: 'tool-calls', raw: 'tool_calls' }, usage: id === 'tool-1' ? firstStepUsage() : usage(150, 30), }, ]; @@ -252,7 +269,11 @@ function buildFixture(options: MidTurnFixtureOptions = {}): MidTurnFixture { return [ first[0]!, { type: 'text-start', id: 'step1-text' }, - { type: 'text-delta', id: 'step1-text', delta: 'ASSISTANT_SENTINEL step one reasoning' }, + { + type: 'text-delta', + id: 'step1-text', + delta: 'ASSISTANT_SENTINEL step one reasoning', + }, { type: 'text-end', id: 'step1-text' }, ...first.slice(1), ]; @@ -275,7 +296,11 @@ function buildFixture(options: MidTurnFixtureOptions = {}): MidTurnFixture { if (call === 3) recordedAtThirdRequest = recorded.length > 0; const chunks = chunksForCall(call); return { - stream: simulateReadableStream({ chunks, initialDelayInMs: null, chunkDelayInMs: null }), + stream: simulateReadableStream({ + chunks, + initialDelayInMs: null, + chunkDelayInMs: null, + }), }; }, }); @@ -446,7 +471,18 @@ function buildFixture(options: MidTurnFixtureOptions = {}): MidTurnFixture { ...(options.providerNative ? { slug: 'codex-subscription', providerType: 'openai-codex' as const } : {}), - models: [{ id: 'mock-model-id', ...(options.withoutContextWindow ? {} : { contextWindow }) }], + models: [ + { + id: 'mock-model-id', + ...(options.withoutContextWindow ? {} : { contextWindow }), + ...(options.modelMaxOutputTokens !== undefined + ? { maxOutputTokens: options.modelMaxOutputTokens } + : {}), + }, + ], + ...(options.withoutContextWindow || options.declareContextWindow === false + ? {} + : { relayModelProfiles: { 'mock-model-id': { contextWindow } } }), }, apiKey: 'sk-test', modelId: 'mock-model-id', @@ -503,27 +539,23 @@ function buildFixture(options: MidTurnFixtureOptions = {}): MidTurnFixture { : {}), ...(options.systemPromptChars ? { systemPrompt: 'S'.repeat(options.systemPromptChars) } : {}), contextBudget: options.useRuntimeDefaultPolicy - ? buildDefaultContextBudgetPolicy( - { - ...connection(), - models: [ - { id: 'mock-model-id', ...(options.withoutContextWindow ? {} : { contextWindow }) }, - ], - }, - { - name: 'runtime-default-mid-turn', - modelId: 'mock-model-id', - }, - ) + ? buildDefaultContextBudgetPolicy({ + name: 'runtime-default-mid-turn', + modelId: 'mock-model-id', + }) : { name: 'mid-turn-test', - maxHistoryEstimatedTokens: options.maxHistoryEstimatedTokens ?? 100_000, historyCompact: { enabled: true, - midTurn: { enabled: true, reserveTokens }, + midTurn: { enabled: true }, }, ...(options.activeToolResultPrune - ? { activeToolResultPrune: { enabled: true, maxCurrentResultEstimatedTokens: 30 } } + ? { + activeToolResultPrune: { + enabled: true, + maxCurrentResultEstimatedTokens: 30, + }, + } : {}), }, ...(options.activeToolResultPrune @@ -586,7 +618,10 @@ function buildFixture(options: MidTurnFixtureOptions = {}): MidTurnFixture { memoryExtraction: { gate: async () => options.memoryGate ?? { allowed: true as const }, automaticGate: () => options.memoryGate ?? { allowed: true as const }, - remember: async () => ({ status: 'unavailable' as const, requestedItems: [] }), + remember: async () => ({ + status: 'unavailable' as const, + requestedItems: [], + }), extract: (snapshot: MemoryExtractionSourceSnapshot) => { memorySnapshots.push(snapshot); return new Promise(() => {}); @@ -681,7 +716,43 @@ function compactionDecisions( } function defineMidTurnSuite(consumer: ConsumerMode): void { - test('compacts over the high water, persists first, and continues the same turn', async () => { + test('keeps user_stop when stopping while a usage-triggered fold is summarizing', async () => { + // The fold now starts from the provider's real usage instead of a local + // estimate, but a user stop during its summarizer call must still end the + // turn as user_stop — not as a summarizer failure and not as a completed + // turn. + let markSummaryStarted: (() => void) | undefined; + const summaryStarted = new Promise((resolve) => { + markSummaryStarted = resolve; + }); + let finishSummary: (() => void) | undefined; + const fixture = buildFixture({ + summarize: () => + new Promise((resolve) => { + finishSummary = () => resolve(undefined); + markSummaryStarted?.(); + }), + }); + const turn = runFixtureTurn(fixture, consumer); + await summaryStarted; + await fixture.backend.stop('user_stop'); + finishSummary?.(); + await turn; + + // The stop landed while the summarizer was running: nothing was persisted, + // no further tool ran, and the one request attempted after the stop was + // rejected by the transport on its already-aborted signal. + assert.equal(fixture.recorded.length, 0); + assert.deepEqual(fixture.toolExecutions, ['one.md', 'two.md']); + assert.equal( + fixture.events.some((event) => event.type === 'abort'), + true, + ); + const complete = fixture.events.find((event) => event.type === 'complete'); + assert.equal(complete?.type === 'complete' ? complete.stopReason : undefined, 'user_stop'); + }); + + test('compacts after provider usage crosses the declared window, persists first, and continues the same turn', async () => { const fixture = buildFixture(); await runFixtureTurn(fixture, consumer); @@ -698,7 +769,10 @@ function defineMidTurnSuite(consumer: ConsumerMode): void { assert.equal(fixture.recordedBeforeThirdRequest(), true); const checkpoint = fixture.recorded[0]!; assert.equal(checkpoint.phase, 'mid_turn'); - assert.deepEqual(checkpoint.headAnchor, { runtimeEventId: 'anchor-1', turnId: 'turn-1' }); + assert.deepEqual(checkpoint.headAnchor, { + runtimeEventId: 'anchor-1', + turnId: 'turn-1', + }); // Coverage: [prior-user, prior-model, anchor, call-1, result-1] — all of // them durable in the ledger before the checkpoint was recorded. assert.equal(checkpoint.coverage.eventCount, 5); @@ -725,17 +799,6 @@ function defineMidTurnSuite(consumer: ConsumerMode): void { assert.equal(midTurnDecision?.decision, 'replaced'); assert.equal(midTurnDecision?.reason, 'context_limit'); assert.deepEqual(midTurnDecision?.boundaryIds, [checkpoint.checkpointId]); - - // Invariant: a persisted checkpoint always passes the single replay gate - // under the same policy the backend replays with — the next projection - // selects it (no coverage_miss, no size rejection). - const fit = evaluateHistoryCompactCheckpointReplay( - checkpoint, - fixture.ledger, - undefined, - 100_000, - ); - assert.equal(fit.fits, true); }); test('replays a Codex V2 mid-turn checkpoint as native provider state', async () => { @@ -839,9 +902,8 @@ function defineMidTurnSuite(consumer: ConsumerMode): void { // Recovery: re-project prior turns + the durable current-turn ledger with // normal thresholds — the checkpoint replays and the covered raw span is - // never re-injected, even though the raw history is below the high water. + // never re-injected, even though the raw history is otherwise small. const replay = applyRuntimeEventContextBudget([...fixture.priorEvents, ...fixture.ledger], { - maxHistoryEstimatedTokens: 100_000, historyCompact: { enabled: true, checkpoint }, }); @@ -864,7 +926,6 @@ function defineMidTurnSuite(consumer: ConsumerMode): void { // fits, so it goes out and the turn runs to its own end. const fixture = buildFixture({ contextWindow: 120, - reserveTokens: 100, withoutPriorTurns: true, }); await runFixtureTurn(fixture, consumer); @@ -889,8 +950,7 @@ function defineMidTurnSuite(consumer: ConsumerMode): void { test('a summary that cannot fit its own input fails open and still dispatches', async () => { const fixture = buildFixture({ - contextWindow: 150, - reserveTokens: 100, + contextWindow: 100, useRuntimeDefaultPolicy: true, summarize: () => { throw new HistoryCompactSummarizerError('input_too_large'); @@ -938,7 +998,9 @@ function defineMidTurnSuite(consumer: ConsumerMode): void { finishReason: 'stop', }), }); - const fixture = buildFixture({ summarize: (input) => malformedSummarize(input) }); + const fixture = buildFixture({ + summarize: (input) => malformedSummarize(input), + }); await runFixtureTurn(fixture, consumer); // The turn still completes on the raw projection; nothing durable claims @@ -972,6 +1034,68 @@ function defineMidTurnSuite(consumer: ConsumerMode): void { assert.equal(complete?.type === 'complete' ? complete.stopReason : undefined, 'end_turn'); }); + test('bounds a provider-error summarizer failure across later steps in the same turn', async () => { + // The baseline that fired this trigger survives a fail-open, so without a + // latch every later step re-evaluates it and dispatches the same doomed + // call. Live evidence: 15 consecutive failed summarizer calls over ~47 + // minutes on a provider that answers slowly and fails (#4634). + const fixture = buildFixture({ + rollingOverflow: true, + summarize: () => { + throw new HistoryCompactSummarizerError('provider_error'); + }, + }); + + await runFixtureTurn(fixture, consumer); + + assert.equal(fixture.summarizerCalls, 1); + assert.equal(fixture.recorded.length, 0); + const failedOpen = compactionDecisions(fixture).filter( + (decision) => decision.decision === 'failedOpen', + ); + assert.ok(failedOpen.length >= 1); + assert.equal(failedOpen[0]?.failOpenReason, 'provider_error'); + }); + + test('does not fold when the reply was cut by the output budget Maka itself sends', async () => { + // `finishReason: length` at exactly the configured `maxOutputTokens` is + // Maka's own cap, not the provider running out of context; the next + // request carries the same cap, so folding would shrink history every + // step without touching the constraint (#4559). + const fixture = buildFixture({ + withoutContextWindow: true, + finalAtSecondCall: true, + firstStepFinishReason: 'length', + modelMaxOutputTokens: 20, + firstStepUsage: { input: 100, output: 20 }, + }); + await runFixtureTurn(fixture, consumer); + + assert.equal(fixture.summarizerCalls, 0); + }); + + test('does not report provider dropping when the step dropped its tool schemas', async () => { + // A finalization step resolves an empty tool set, so its request loses + // several thousand schema tokens with no fold, prune or image omission. + // Maka shaped that request; the provider dropped nothing. + const fixture = buildFixture({ + contextWindow: 200, + finalAtSecondCall: true, + childFinalization: true, + finalStepUsage: { input: 50, output: 10 }, + }); + await runFixtureTurn(fixture, consumer); + + assert.equal( + fixture.messages.some( + (message) => + (message as { type?: string; kind?: string }).type === 'system_note' && + (message as { kind?: string }).kind === 'context_provider_dropping', + ), + false, + ); + }); + test('fails closed before provider dispatch when the durable ledger read fails', async () => { const fixture = buildFixture(); // Break the seam after construction: every trigger read now rejects. @@ -1026,8 +1150,7 @@ function defineMidTurnSuite(consumer: ConsumerMode): void { // AFTER the capacity hook) archives the result down to a placeholder that // fits the window. const fixture = buildFixture({ - contextWindow: 500, - reserveTokens: 100, + contextWindow: 100, withoutPriorTurns: true, hugeFirstResult: true, finalAtSecondCall: true, @@ -1055,8 +1178,7 @@ function defineMidTurnSuite(consumer: ConsumerMode): void { // estimate must count it — without that the 500-token window is never // crossed and the trigger never fires at all. const fixture = buildFixture({ - contextWindow: 500, - reserveTokens: 100, + contextWindow: 100, withoutPriorTurns: true, bigToolGroup: true, }); @@ -1069,95 +1191,186 @@ function defineMidTurnSuite(consumer: ConsumerMode): void { assert.equal(failedOpen?.failOpenReason, 'no_safe_completed_span'); }); - test('a fold that cannot shrink the real payload is refused, not applied (runaway summary)', async () => { - // The summarizer returns a block far larger than the span it replaces. - // Applying it would hand the verdict owner a WORSE request than the raw - // projection; the hook measures the materialized payload and keeps the raw - // messages instead. Validation runs before the recorder, so the rejected - // checkpoint is never persisted (asserted below). + test("the usage baseline is the last request's input plus output", async () => { + // Baseline 120 (100 + 20) with a 40-token reserve (twice the 20-token + // reply). A window of 155 is crossed only because the baseline counts the + // output: input alone plus the reserve is 140, which fits. + const compacting = buildFixture({ + contextWindow: 155, + finalAtSecondCall: true, + firstStepUsage: { input: 100, output: 20 }, + }); + await runFixtureTurn(compacting, consumer); + + assert.equal(compacting.summarizerCalls, 1); + assert.match(promptJson(compacting, 1), /maka_history_compact_checkpoint/); + + const fitting = buildFixture({ + contextWindow: 165, + finalAtSecondCall: true, + firstStepUsage: { input: 100, output: 20 }, + }); + await runFixtureTurn(fitting, consumer); + + assert.equal(fitting.summarizerCalls, 0); + }); + + test('reports an accepted request past the window the model reports when nothing is declared', async () => { + // The kimi k3-256k shape: the provider accepts past its own reported + // window without rejecting or truncating, so no other signal fires. const fixture = buildFixture({ - summarize: () => - `## Goal\n${'GIANT_SUMMARY_'.repeat(600)}\n\n## Progress\n- done\n\n## Next Steps\n1. continue\n\n## Critical Context\n- (none)`, + contextWindow: 100, + declareContextWindow: false, + finalAtSecondCall: true, + firstStepUsage: { input: 150, output: 40 }, }); await runFixtureTurn(fixture, consumer); - assert.equal(fixture.model.doStreamCalls.length, 3); - const complete = fixture.events.find((event) => event.type === 'complete'); - assert.equal(complete?.type === 'complete' ? complete.stopReason : undefined, 'end_turn'); - // The raw span stayed; the giant block was never sent. - const thirdPrompt = promptJson(fixture, 2); - assert.equal(thirdPrompt.includes('RAW_SPAN_ONE_'), true); - assert.equal(thirdPrompt.includes('GIANT_SUMMARY_'), false); - const failedOpen = compactionDecisions(fixture).find( - (decision) => decision.phase === 'mid_turn' && decision.decision === 'failedOpen', + const note = fixture.messages.find( + (message): message is { type: 'system_note'; kind: string; data?: unknown } => + (message as { kind?: string }).kind === 'context_reported_window_exceeded', ); - assert.equal(failedOpen?.failOpenReason, 'replacement_not_smaller'); - // Review round-4 finding 3: a checkpoint whose replacement was REJECTED - // must never be persisted — replay applies the session's latest checkpoint - // before any high-water check, so a persisted runaway block would poison - // every later projection even though this step correctly refused it. - assert.equal(fixture.recorded.length, 0); + assert.deepEqual(note?.data, { usedTokens: 190, reportedContextWindow: 100 }); + // A hint is still not a declaration: nothing folded on its own. + assert.equal(fixture.summarizerCalls, 0); }); - test("the usage baseline is the last request's INPUT tokens — output is not double-counted (review finding 1)", async () => { - // Review round-4 finding 1 repro shape: a step with heavy output. The - // signed payload delta already carries the freshly generated assistant - // output and tool results, so a baseline of input+output counts the - // output twice (~500 real tokens estimated as ~900) and terminates a - // turn that actually fits the window. + test('does not repeat the reported-window note once the session is already past the line', async () => { + // On these providers usage keeps growing past the reported window, so the + // note fires on the crossing, not on every send. A persisted anchor that + // is already over the line carries that across sessions. const fixture = buildFixture({ - contextWindow: 500, - reserveTokens: 100, - withoutPriorTurns: true, + contextWindow: 100, + declareContextWindow: false, finalAtSecondCall: true, - firstStepUsage: { input: 300, output: 380 }, + firstStepUsage: { input: 150, output: 40 }, + extraPriorEvents: [priorUsageEvent({ inputTokens: 300, outputTokens: 20 })], + priorRunHeaders: [priorRunHeader()], }); await runFixtureTurn(fixture, consumer); - // input(300) + payload delta (~hundred tokens) fits the 500 window; the - // double-counting baseline (680 + delta) would have exhausted it. - const complete = fixture.events.find((event) => event.type === 'complete'); - assert.equal(complete?.type === 'complete' ? complete.stopReason : undefined, 'end_turn'); - assert.equal(fixture.model.doStreamCalls.length, 2); + assert.equal( + fixture.messages.some( + (message) => (message as { kind?: string }).kind === 'context_reported_window_exceeded', + ), + false, + ); }); - test('a usage object without usable input tokens falls back to cold start, never to zero (review finding 1)', async () => { - // Reverse direction: the adapter normalizes missing token fields to 0. A - // zero baseline plus a small delta estimates a huge request as tiny and - // lets it stream over the window; an unusable usage sample must instead - // fall back to the whole-payload cold-start estimate, which triggers the - // fold here (big priors leave a safe span, so compaction rescues). + test('does not report the reported window when the user declared one', async () => { + // With a declaration the overrun note owns this case; two notes for one + // fact would be noise. const fixture = buildFixture({ - contextWindow: 1_000, - reserveTokens: 100, - priorChars: 2_000, + contextWindow: 100, finalAtSecondCall: true, - firstStepUsage: 'missing', + firstStepUsage: { input: 150, output: 40 }, }); await runFixtureTurn(fixture, consumer); - assert.equal(fixture.recorded.length, 1); - const secondPrompt = promptJson(fixture, 1); - assert.match(secondPrompt, /maka_history_compact_checkpoint/); - assert.equal(secondPrompt.includes('PRIOR_FACT'), false); - const complete = fixture.events.find((event) => event.type === 'complete'); - assert.equal(complete?.type === 'complete' ? complete.stopReason : undefined, 'end_turn'); + assert.equal( + fixture.messages.some( + (message) => (message as { kind?: string }).kind === 'context_reported_window_exceeded', + ), + false, + ); }); - test('a runaway summary is rejected as replacement_not_smaller and never persisted', async () => { + test('reports a reply that ran past the declared window', async () => { const fixture = buildFixture({ - contextWindow: 150, - reserveTokens: 100, - summarize: () => - `## Goal\n${'GIANT_SUMMARY_'.repeat(600)}\n\n## Progress\n- done\n\n## Next Steps\n1. continue\n\n## Critical Context\n- (none)`, + contextWindow: 200, + finalAtSecondCall: true, + firstStepUsage: { input: 150, output: 60 }, }); await runFixtureTurn(fixture, consumer); - const failedOpen = compactionDecisions(fixture).find( - (decision) => decision.phase === 'mid_turn' && decision.decision === 'failedOpen', + const note = fixture.messages.find( + (message): message is { type: 'system_note'; kind: string; data?: unknown } => + (message as { type?: string }).type === 'system_note' && + (message as { kind?: string }).kind === 'context_window_overrun', + ); + assert.deepEqual(note?.data, { usedTokens: 210, declaredContextWindow: 200 }); + }); + + test('does not report an overrun for an exchange that stayed inside the window', async () => { + const fixture = buildFixture({ + contextWindow: 200, + finalAtSecondCall: true, + firstStepUsage: { input: 150, output: 40 }, + }); + await runFixtureTurn(fixture, consumer); + + assert.equal( + fixture.messages.some( + (message) => (message as { kind?: string }).kind === 'context_window_overrun', + ), + false, ); - assert.equal(failedOpen?.failOpenReason, 'replacement_not_smaller'); - assert.equal(fixture.recorded.length, 0); + }); + + test('records provider context dropping when an append-only step reports the same usage', async () => { + // The Ollama shape: the provider truncates to its own window, so input + // stops growing rather than dropping while Maka keeps appending. A + // plateau is the signal the copy promises ("usage did not grow"). + const fixture = buildFixture({ + contextWindow: 200, + finalAtSecondCall: true, + firstStepUsage: { input: 100, output: 20 }, + finalStepUsage: { input: 100, output: 10 }, + }); + await runFixtureTurn(fixture, consumer); + + const note = fixture.messages.find( + (message): message is { type: 'system_note'; kind: string } => + (message as { type?: string }).type === 'system_note', + ); + assert.equal(note?.kind, 'context_provider_dropping'); + }); + + test('records provider context dropping only for an unshaped usage decrease', async () => { + const fixture = buildFixture({ + contextWindow: 200, + finalAtSecondCall: true, + finalStepUsage: { input: 50, output: 10 }, + }); + await runFixtureTurn(fixture, consumer); + + const note = fixture.messages.find( + (message): message is { type: 'system_note'; kind: string } => + (message as { type?: string }).type === 'system_note', + ); + assert.equal(note?.kind, 'context_provider_dropping'); + }); + + test('does not call provider context dropping when active pruning explains the decrease', async () => { + const fixture = buildFixture({ + contextWindow: 200, + finalAtSecondCall: true, + hugeFirstResult: true, + activeToolResultPrune: true, + finalStepUsage: { input: 50, output: 10 }, + }); + await runFixtureTurn(fixture, consumer); + + assert.equal( + fixture.messages.some( + (message) => + (message as { type?: string; kind?: string }).type === 'system_note' && + (message as { kind?: string }).kind === 'context_provider_dropping', + ), + false, + ); + }); + + test('folds once after a provider output-limit finish without a window', async () => { + const fixture = buildFixture({ + withoutContextWindow: true, + finalAtSecondCall: true, + firstStepFinishReason: 'length', + }); + await runFixtureTurn(fixture, consumer); + + assert.equal(fixture.summarizerCalls, 1); + assert.match(promptJson(fixture, 1), /maka_history_compact_checkpoint/); }); test("a completed step's assistant text is never dropped from the replacement (review finding B)", async () => { @@ -1175,7 +1388,7 @@ function defineMidTurnSuite(consumer: ConsumerMode): void { const fixture = buildFixture({ // High water at 100 tokens: the first request's 100 input tokens plus // its assistant/tool-result delta crosses it at the step-1 boundary. - reserveTokens: 1_900, + contextWindow: 100, assistantTextInFirstStep: true, finalAtSecondCall: true, }); @@ -1208,7 +1421,10 @@ describe('mid-turn capacity compaction in the streaming backend', () => { defineMidTurnSuite('immediate'); test('dispatches a mid-turn Compaction recipe after persistence without awaiting it', async () => { - const fixture = buildFixture({ captureMemoryExtraction: true, systemPromptChars: 32 }); + const fixture = buildFixture({ + captureMemoryExtraction: true, + systemPromptChars: 32, + }); await runFixtureTurn(fixture); assert.equal(fixture.recorded.length, 1); @@ -1337,62 +1553,6 @@ describe('mid-turn capacity default-on safety guards (issue #882 PR 3)', () => { assert.doesNotMatch(promptJson(fixture, 0), /omitted after provider context overflow/); }); - test('compacts one oversized prior turn before an unknown-model request', async () => { - const fixture = buildFixture({ - useRuntimeDefaultPolicy: true, - withoutContextWindow: true, - priorChars: 180_000, - priorShape: 'tool_heavy', - }); - await runFixtureTurn(fixture); - - assert.equal(fixture.summarizerCalls, 1); - assert.equal(fixture.recorded.length, 1); - assert.equal(fixture.recorded[0]?.coverage.eventCount, 4); - assert.match(fixture.summarizedSources[0] ?? '', /function_call/); - assert.match(fixture.summarizedSources[0] ?? '', /function_response/); - const firstPrompt = promptJson(fixture, 0); - assert.match(firstPrompt, /maka_history_compact_checkpoint/); - assert.match(firstPrompt, /MID_TURN_SUMMARY_SENTINEL/); - assert.equal(firstPrompt.includes('OVERSIZED_TOOL_RESULT_'), false); - const complete = fixture.events.find((event) => event.type === 'complete'); - assert.equal(complete?.type === 'complete' ? complete.stopReason : undefined, 'end_turn'); - }); - - test('dispatches an oversized prior turn when its summary fails, after trying once', async () => { - const fixture = buildFixture({ - useRuntimeDefaultPolicy: true, - withoutContextWindow: true, - priorChars: 180_000, - priorShape: 'tool_heavy', - summarize: () => undefined, - }); - await runFixtureTurn(fixture); - - assert.equal(fixture.summarizerCalls, 1); - assert.equal(fixture.model.doStreamCalls.length > 0, true); - const complete = fixture.events.find((event) => event.type === 'complete'); - assert.equal(complete?.type === 'complete' ? complete.stopReason : undefined, 'end_turn'); - }); - - test('compacts an oversized latest turn when an older turn is also retained', async () => { - const fixture = buildFixture({ - useRuntimeDefaultPolicy: true, - withoutContextWindow: true, - priorChars: 180_000, - priorShape: 'tool_heavy', - }); - fixture.priorEvents.unshift( - runtimeTextEvent('older-user', 'turn-older', 'user', 'older question'), - runtimeTextEvent('older-model', 'turn-older', 'model', 'older answer'), - ); - await runFixtureTurn(fixture); - - assert.equal(fixture.summarizerCalls, 1); - assert.equal(fixture.recorded.length, 1); - assert.equal(promptJson(fixture, 0).includes('OVERSIZED_TOOL_RESULT_'), false); - }); - test('keeps multiple bounded recent turns below the model capacity', async () => { const fixture = buildFixture({ useRuntimeDefaultPolicy: true, @@ -1433,38 +1593,6 @@ describe('mid-turn capacity default-on safety guards (issue #882 PR 3)', () => { assert.equal(complete?.type === 'complete' ? complete.stopReason : undefined, 'end_turn'); }); - test('keeps user_stop when stopping an oversized pre-turn summary', async () => { - let markSummaryStarted: (() => void) | undefined; - const summaryStarted = new Promise((resolve) => { - markSummaryStarted = resolve; - }); - let finishSummary: (() => void) | undefined; - const fixture = buildFixture({ - useRuntimeDefaultPolicy: true, - withoutContextWindow: true, - priorChars: 180_000, - priorShape: 'tool_heavy', - summarize: () => - new Promise((resolve) => { - finishSummary = () => resolve(undefined); - markSummaryStarted?.(); - }), - }); - const turn = runFixtureTurn(fixture); - await summaryStarted; - await fixture.backend.stop('user_stop'); - finishSummary?.(); - await turn; - - assert.equal(fixture.model.doStreamCalls.length, 0); - assert.equal( - fixture.events.some((event) => event.type === 'abort'), - true, - ); - const complete = fixture.events.find((event) => event.type === 'complete'); - assert.equal(complete?.type === 'complete' ? complete.stopReason : undefined, 'user_stop'); - }); - test('does not fire for a session without a persisted head anchor (child sessions have no seam)', async () => { // PR 1's decision: child sessions are structurally without the head-anchor // seam, so even with midTurn on by default the trigger must never activate. @@ -1488,23 +1616,22 @@ describe('mid-turn capacity default-on safety guards (issue #882 PR 3)', () => { assert.equal(complete?.type === 'complete' ? complete.stopReason : undefined, 'end_turn'); }); - test('never runs a pointless summarizer on a small-window model under the shipped defaults (review P2)', async () => { - // gpt-4 shape: an 8192-token window under the runtime-derived defaults. - // A flat 16384 reserve used to clamp the mid_turn high water to 1 token — - // every boundary triggered, the summarizer ran, and the checkpoint could - // never pass the 1-token replay gate: pure waste. With the window-bounded - // reserve the payload sits far below the high water, so the default must - // be completely inert here: no summarizer call, no checkpoint, and the - // turn completes on the raw projection. - const fixture = buildFixture({ useRuntimeDefaultPolicy: true, contextWindow: 8_192 }); - await runFixtureTurn(fixture); + test('treats a /models context window as metadata unless the user declares it', async () => { + const reported = buildFixture({ + useRuntimeDefaultPolicy: true, + contextWindow: 100, + declareContextWindow: false, + }); + await runFixtureTurn(reported); + assert.equal(reported.summarizerCalls, 0); - assert.equal(fixture.summarizerCalls, 0); - assert.equal(fixture.recorded.length, 0); - assert.equal(fixture.model.doStreamCalls.length, 3); - const complete = fixture.events.find((event) => event.type === 'complete'); - assert.equal(complete?.type === 'complete' ? complete.stopReason : undefined, 'end_turn'); - assert.equal(promptJson(fixture, 2).includes('RAW_SPAN_ONE_'), true); + const declared = buildFixture({ + useRuntimeDefaultPolicy: true, + contextWindow: 190, + }); + await runFixtureTurn(declared); + assert.equal(declared.summarizerCalls, 1); + assert.match(promptJson(declared, 2), /maka_history_compact_checkpoint/); }); }); @@ -1513,15 +1640,14 @@ describe('the shipped runtime default drives the proactive long-turn journey (is // No hand-built policy and no env override: this wires // buildDefaultContextBudgetPolicy — the exact default every surface now // inherits — into the backend. A long turn whose real usage crosses - // `contextWindow - derivedReserve` (window 1000 → reserve 250, high water - // 750) must fold a safe completed prefix into a DURABLE mid_turn + // Real provider usage crossing the declared window must fold a safe + // completed prefix into a DURABLE mid_turn // checkpoint and continue the SAME turn to normal completion, never // truncate it or surface a raw provider error. const fixture = buildFixture({ useRuntimeDefaultPolicy: true, - contextWindow: 1_000, + contextWindow: 190, priorChars: 1_400, - firstStepUsage: { input: 900, output: 20 }, }); await runFixtureTurn(fixture); @@ -1530,7 +1656,7 @@ describe('the shipped runtime default drives the proactive long-turn journey (is assert.equal(fixture.recorded[0]?.phase, 'mid_turn'); // The continued request rides the compacted projection: the compact block // is present and the replaced raw span is gone. - const continuedPrompt = promptJson(fixture, 1); + const continuedPrompt = promptJson(fixture, 2); assert.match(continuedPrompt, /maka_history_compact_checkpoint/); assert.match(continuedPrompt, /MID_TURN_SUMMARY_SENTINEL/); assert.equal(continuedPrompt.includes('PRIOR_FACT'), false); @@ -1546,165 +1672,117 @@ describe('the shipped runtime default drives the proactive long-turn journey (is ); }); - test('persists the LAST request as the anchor while input stays the send sum', async () => { - // `input` is the reconciled per-send sum (#996) and anchors nothing: an - // estimate started from it would be off by every earlier step. The - // persisted anchor is the last request alone, paired with the payload - // chars measured for that same request. - const fixture = buildFixture({ contextWindow: 1_000_000, finalAtSecondCall: true }); + test('persists the last request input and output as the anchor', async () => { + const fixture = buildFixture({ + contextWindow: 1_000_000, + finalAtSecondCall: true, + }); await runFixtureTurn(fixture); const usage = fixture.messages.find( - (message): message is { type: 'token_usage'; input: number; lastRequestAnchor?: unknown } => - (message as { type?: string }).type === 'token_usage', + ( + message, + ): message is { + type: 'token_usage'; + input: number; + lastRequestAnchor?: unknown; + } => (message as { type?: string }).type === 'token_usage', ); // Two steps: 100 + 120 reported input, and the send sum is both. assert.equal(usage?.input, 220); const anchor = usage?.lastRequestAnchor as - | { inputTokens: number; payloadChars: number } + | { inputTokens: number; outputTokens?: number } | undefined; assert.equal(anchor?.inputTokens, 120); - assert.equal((anchor?.payloadChars ?? 0) > 0, true); + assert.equal(anchor?.outputTokens, 10); }); - /** - * Turn start has exactly ONE trigger. These rows differ only in what the - * prior context offers — a persisted anchor, an armed pre-turn gate, both, - * neither — and each names the one thing that may act on it. The gate weighs - * a subset of the payload with a cruder ruler, so it is a fallback, never a - * parallel authority. - */ - for (const row of [ - { - name: 'a persisted anchor compacts the turn’s FIRST request', - priorAnchor: 'previous_turn', - armGate: false, - foldedBy: 'anchored_estimate', - }, - { - name: 'neither armed leaves the first request alone', - priorAnchor: 'none', - armGate: false, - foldedBy: 'nothing', - }, - { - name: 'the anchored estimate acts while the armed gate stands down', - priorAnchor: 'previous_turn', - armGate: true, - foldedBy: 'anchored_estimate', - }, - { - name: 'without an anchor the armed gate shapes the oversized history', - priorAnchor: 'none', - armGate: true, - foldedBy: 'pre_turn_gate', - }, - { - // A manual /compact writes `input: 0, output: 0` without going through - // the provider send, so it carries no anchor: the reverse scan skips it - // and still finds the last real request. - name: 'a synthetic /compact usage row does not shadow the real anchor', - priorAnchor: 'behind_compact_row', - armGate: false, - foldedBy: 'anchored_estimate', - }, - ] as const) { - test(`turn start: ${row.name}`, async () => { - // High water 20k: the whole payload at chars/4 is nowhere near it, so - // without an anchor step 0 has nothing to act on. The anchor says the - // request really costs 30k. + test('an anchor is discarded unless a run header proves it came from this model', async () => { + // Input tokens are a count in one model's tokenizer; nothing converts them. + // The anchor sits ABOVE the declared window, so it is the header check + // alone that decides: a matching header folds at step 0, while a header + // naming another model and no header at all leave the request alone. + const anchor = priorUsageEvent({ inputTokens: 30_000, outputTokens: 10 }); + for (const [priorRunHeaders, folds] of [ + [[priorRunHeader()], true], + [[{ ...priorRunHeader(), modelId: 'some-other-model' }], false], + [[], false], + ] as const) { const fixture = buildFixture({ priorChars: 2_000, - contextWindow: 40_000, - reserveTokens: 20_000, + contextWindow: 20_000, finalAtSecondCall: true, - ...(row.armGate ? { maxHistoryEstimatedTokens: 400 } : {}), - extraPriorEvents: await priorAnchorEvents(row.priorAnchor), - ...(row.priorAnchor === 'none' ? {} : { priorRunHeaders: [priorRunHeader()] }), + extraPriorEvents: [anchor], + priorRunHeaders: [...priorRunHeaders], }); await runFixtureTurn(fixture); - const decisions = compactionDecisions(fixture); - const anchoredFold = decisions.find( - (decision) => decision.stage === 'activeStep' && decision.decision === 'replaced', - ); - assert.equal(anchoredFold !== undefined, row.foldedBy === 'anchored_estimate'); - assert.equal( - decisions.some((decision) => decision.stage === 'priorReplay'), - row.foldedBy === 'pre_turn_gate', - ); - assert.equal(fixture.recorded.length, row.foldedBy === 'nothing' ? 0 : 1); - - const firstPrompt = promptJson(fixture, 0); - if (row.foldedBy === 'nothing') { - assert.equal(firstPrompt.includes('PRIOR_FACT'), true); - return; - } - assert.match(firstPrompt, /maka_history_compact_checkpoint/); - assert.equal(firstPrompt.includes('PRIOR_FACT'), false); - assert.equal(firstPrompt.includes(ANCHOR_TEXT), true); - if (row.foldedBy !== 'anchored_estimate') return; - // The first request folds on the pre_turn boundary: the head anchor stays - // verbatim in the successor tail instead of being covered. - assert.equal(anchoredFold?.phase, 'pre_turn'); - assert.match(firstPrompt, /MID_TURN_SUMMARY_SENTINEL/); - }); - } + assert.equal(fixture.recorded.length, folds ? 1 : 0); + } + }); - test('the anchor a finalization step writes excludes the tool schemas it cleared', async () => { - // The child-summary finalization step sends no tools at all. Measuring the - // pre-dispatch tool set would pair the provider's real input count with a - // payload 12k chars larger than the request it counted. - const finalization = buildFixture({ - contextWindow: 1_000_000, - finalAtSecondCall: true, - bigActiveTool: true, - childFinalization: true, - }); - await runFixtureTurn(finalization); - const control = buildFixture({ - contextWindow: 1_000_000, + test('a synthetic /compact usage row does not shadow the real anchor', async () => { + // A manual /compact writes `input: 0, output: 0` without a provider send, + // so it carries no anchor: the reverse scan skips it and still finds the + // last real request, which is above the window and folds step 0. + const fixture = buildFixture({ + priorChars: 2_000, + contextWindow: 20_000, finalAtSecondCall: true, - bigActiveTool: true, + extraPriorEvents: [ + priorUsageEvent({ inputTokens: 30_000, outputTokens: 10 }), + { + ...runtimeTextEvent('prior-compact-usage', 'turn-0', 'model', ''), + id: 'prior-compact-usage', + runId: 'run-0', + invocationId: 'run-0', + role: 'system' as const, + author: 'system' as const, + content: undefined, + actions: { tokenUsage: { input: 0, output: 0 } }, + }, + ], + priorRunHeaders: [priorRunHeader()], }); - await runFixtureTurn(control); + await runFixtureTurn(fixture); - assert.equal( - promptJson(finalization, 1).includes('BIG_ACTIVE_SCHEMA'), - false, - 'the finalization request itself carries no tool schema', - ); - assert.equal(anchorOf(control) !== undefined, true); - assert.equal( - (anchorOf(control)?.payloadChars ?? 0) > BIG_ACTIVE_TOOL_SCHEMA_CHARS, - true, - 'an ordinary last request does carry the schema', + assert.equal(fixture.recorded.length, 1); + const fold = compactionDecisions(fixture).find( + (decision) => decision.stage === 'activeStep' && decision.decision === 'replaced', ); - assert.equal((anchorOf(finalization)?.payloadChars ?? 0) < BIG_ACTIVE_TOOL_SCHEMA_CHARS, true); + assert.equal(fold?.phase, 'pre_turn'); }); - test('an anchor is discarded unless a run header proves it came from this model', async () => { - // Input tokens are a count in one model's tokenizer; nothing converts them. - // A header naming another model and no header at all fail the same way. - for (const priorRunHeaders of [[{ ...priorRunHeader(), modelId: 'some-other-model' }], []]) { + test('the reserve is twice the last real reply, bounded, not the model output limit', async () => { + // With the window declared at the provider's real size, an accepted + // request can never exceed it on its own; the reply the next request must + // leave room for is what tips it. That room is measured from the reply the + // model actually wrote, so a session whose answers are long reserves more + // than one whose answers are short, and neither number is a guess. The + // model's own output limit is deliberately not the reserve: on k3-256k it + // is half the window and would fold at 50% utilization (#4634). + for (const [anchorOutput, folds] of [ + [60, true], + [5, false], + ] as const) { const fixture = buildFixture({ - priorChars: 2_000, - contextWindow: 40_000, - reserveTokens: 20_000, + priorChars: 200, + contextWindow: 1_000, finalAtSecondCall: true, - extraPriorEvents: [priorUsageEvent({ inputTokens: 30_000, payloadChars: 4_000 })], - priorRunHeaders, + modelMaxOutputTokens: 600, + extraPriorEvents: [priorUsageEvent({ inputTokens: 900, outputTokens: anchorOutput })], + priorRunHeaders: [priorRunHeader()], }); await runFixtureTurn(fixture); - - assert.equal(fixture.recorded.length, 0); + // 960 + 120 crosses 1,000; 905 + 10 does not. The 600-token output limit + // is irrelevant to both. + assert.equal(fixture.summarizerCalls, folds ? 1 : 0); } }); test('an unrescuable turn under the shipped default still dispatches', async () => { - // Same runtime-derived default (window 120 → reserve 30, high water 90): - // no prior turns leaves no safe completed span, and the request genuinely - // exceeds the local window — which only the provider can act on. + // No prior turns leaves no safe completed span. The request still goes out + // because only the provider can decide whether it fits. const fixture = buildFixture({ useRuntimeDefaultPolicy: true, contextWindow: 120, @@ -1727,67 +1805,9 @@ describe('the shipped runtime default drives the proactive long-turn journey (is * so the next fixture turn sees them the way `prior-run-context` serves them * back after a restart. */ -/** The `lastRequestAnchor` the fixture's turn persisted on its usage message. */ -function anchorOf( - fixture: MidTurnFixture, -): { inputTokens: number; payloadChars: number } | undefined { - const usage = fixture.messages.find( - (message): message is { type: 'token_usage'; lastRequestAnchor?: unknown } => - (message as { type?: string }).type === 'token_usage', - ); - return usage?.lastRequestAnchor as { inputTokens: number; payloadChars: number } | undefined; -} - -/** - * The prior-turn events one turn-start row starts from. `previous_turn` runs a - * whole turn first and takes what the backend actually persisted, so nothing - * hand-builds the anchor it then reads back. - */ -async function priorAnchorEvents( - kind: 'none' | 'previous_turn' | 'behind_compact_row', -): Promise { - if (kind === 'none') return []; - if (kind === 'behind_compact_row') { - return [ - priorUsageEvent({ inputTokens: 30_000, payloadChars: 4_000 }), - { - ...runtimeTextEvent('prior-compact-usage', 'turn-0', 'model', ''), - id: 'prior-compact-usage', - runId: 'run-0', - role: 'system' as const, - author: 'system' as const, - content: undefined, - actions: { tokenUsage: { input: 0, output: 0 } }, - }, - ]; - } - const previous = buildFixture({ - priorChars: 2_000, - contextWindow: 1_000_000, - finalAtSecondCall: true, - // A dense (CJK-shaped) request: real input tokens well above chars/4. - finalStepUsage: { input: 30_000, output: 10 }, - }); - await runFixtureTurn(previous); - const persisted = priorTurnUsageEvents(previous); - assert.equal(persisted.length, 1); - assert.equal( - (persisted[0]?.actions?.tokenUsage?.lastRequestAnchor as { inputTokens: number } | undefined) - ?.inputTokens, - 30_000, - ); - return persisted; -} - -function priorTurnUsageEvents(fixture: MidTurnFixture): RuntimeEvent[] { - return fixture.ledger - .filter((event) => event.actions?.tokenUsage !== undefined) - .map((event) => ({ ...event, turnId: 'turn-0', runId: 'run-0', invocationId: 'run-0' })); -} - function priorUsageEvent(lastRequestAnchor: { inputTokens: number; - payloadChars: number; + outputTokens?: number; }): RuntimeEvent { return { ...runtimeTextEvent('prior-usage', 'turn-0', 'model', ''), diff --git a/packages/runtime/src/__tests__/overflow-reactive-recovery.test.ts b/packages/runtime/src/__tests__/overflow-reactive-recovery.test.ts index b5066a0b04..ecdcca2696 100644 --- a/packages/runtime/src/__tests__/overflow-reactive-recovery.test.ts +++ b/packages/runtime/src/__tests__/overflow-reactive-recovery.test.ts @@ -128,12 +128,12 @@ const BIG_RESULT = 'BIG_RESULT_'.repeat(200); interface ReactiveFixtureOptions { script: CallKind[]; contextWindow?: number; + declareContextWindow?: boolean; /** * A model that declares no context window, on a provider whose default * policy sets no history budget either — so nothing can synthesize one. */ withoutContextWindow?: boolean; - reserveTokens?: number; midTurnEnabled?: boolean; withoutPriorTurns?: boolean; bigPriors?: boolean; @@ -206,6 +206,7 @@ interface ReactiveFixture { priorEvents: RuntimeEvent[]; priorRunHeaders: AgentRunHeader[]; events: SessionEvent[]; + messages: unknown[]; llmCalls: ReactiveLlmCall[]; /** Canonical settlements, whole, when `canonicalAccounting` is on. */ commits: ModelCallCommit[]; @@ -217,11 +218,11 @@ interface ReactiveFixture { function buildReactiveFixture(options: ReactiveFixtureOptions): ReactiveFixture { const contextWindow = options.contextWindow ?? 200_000; - const reserveTokens = options.reserveTokens ?? 1_000; const recorded: HistoryCompactCheckpoint[] = []; const commits: ModelCallCommit[] = []; const toolExecutions: string[] = []; const events: SessionEvent[] = []; + const messages: unknown[] = []; const llmCalls: ReactiveLlmCall[] = []; const retryDelays: number[] = []; const counters = { summarizerCalls: 0 }; @@ -578,7 +579,8 @@ function buildReactiveFixture(options: ReactiveFixtureOptions): ReactiveFixture const backend = createTestAiSdkBackend({ sessionId: 'session-1', header: header(), - appendMessage: async () => { + appendMessage: async (message) => { + messages.push(message); if (!options.slowAppendMessage) return; for (let i = 0; i < 5; i += 1) await flushMacrotask(); }, @@ -593,6 +595,9 @@ function buildReactiveFixture(options: ReactiveFixtureOptions): ReactiveFixture ? { id: 'mock-model-id' } : { id: 'mock-model-id', contextWindow }, ], + ...(options.declareContextWindow + ? { relayModelProfiles: { 'mock-model-id': { contextWindow } } } + : {}), }, apiKey: 'sk-test', ...(options.reasoningReplayTail ? { providerStateIdentity: PROVIDER_STATE_IDENTITY } : {}), @@ -650,10 +655,9 @@ function buildReactiveFixture(options: ReactiveFixtureOptions): ReactiveFixture name: 'reactive-test', // An undeclared window on this provider carries no history budget either, // matching what the default policy builds for it. - ...(options.withoutContextWindow ? {} : { maxHistoryEstimatedTokens: 100_000 }), historyCompact: { enabled: true, - ...(midTurnEnabled ? { midTurn: { enabled: true, reserveTokens } } : {}), + ...(midTurnEnabled ? { midTurn: { enabled: true } } : {}), }, ...(options.activeToolResultPrune ? { activeToolResultPrune: { enabled: true, maxCurrentResultEstimatedTokens: 100 } } @@ -711,6 +715,7 @@ function buildReactiveFixture(options: ReactiveFixtureOptions): ReactiveFixture priorEvents, priorRunHeaders, events, + messages, llmCalls, commits, retryDelays, @@ -1494,13 +1499,9 @@ describe('reactive overflow recovery in the streaming backend', () => { test('the recovery baseline is the request the provider rejected, not the attempt-initial messages', async () => { // Review P1-1 repro: four completed tool steps grow the provider-visible - // request far beyond the attempt's INITIAL messages. The fold shrinks the - // real rejected request but is larger than that initial request, so a - // baseline anchored to the initial messages refuses it as - // replacement_not_smaller and the turn dies on the exact scenario reactive - // recovery exists for — same-turn tool growth. The unique baseline owner - // is the verdict owner's per-request payload measure of the request that - // actually went out. + // request far beyond the attempt's INITIAL messages. Recovery must fold the + // durable rejected-request history rather than relying on that stale base; + // same-turn tool growth must remain recoverable. const fixture = buildReactiveFixture({ script: ['tool', 'tool', 'tool', 'tool', 'overflow', 'done'], }); @@ -1552,7 +1553,7 @@ describe('reactive overflow recovery in the streaming backend', () => { test("a completed retry step's assistant text is never dropped by a post-retry compaction (review P1-A)", async () => { // Review round-2 P1-A repro: provider request boundaries and the Runtime's - // flushedSteps / replacedStepNumber / lastShapeFailure are send-level. + // flushedSteps / replacedStepNumber state are send-level. // After a retry, a mismatched request-local boundary could satisfy the // durability wait before the retry step's text_complete is durable, and // because the @@ -1565,14 +1566,11 @@ describe('reactive overflow recovery in the streaming backend', () => { // already pushed — so `consumed >= pushed` holds and only the send-global // flushedSteps bound can still hold the ledger read back. const fixture = buildReactiveFixture({ - // High water 400: the first attempt's boundary (~usage 100 + small - // delta) stays under it, the retry step's huge result (~BIG_RESULT/4) - // crosses it, so the capacity trigger fires exactly at the retry's own - // step boundary. + // The first attempt's boundary stays below the declared target, while + // the retry step's large result exercises the next request boundary. script: ['tool', 'overflow', 'bigtool', 'done'], bigPriors: true, contextWindow: 2_000, - reserveTokens: 1_600, slowAppendMessage: true, }); await runTurn(fixture); @@ -1785,14 +1783,11 @@ describe('reactive overflow recovery in the streaming backend', () => { }); test('a checkpoint fold never covers an injected steering message', async () => { - // Round-5 F2: injected steering is PINNED out of the foldable span, so a - // measurement of the folded request is a measurement of what the provider - // actually receives — the accumulator re-appends the directive either way. - // + // Round-5 F2: injected steering is PINNED out of the foldable span, so the + // accumulator re-appends the directive exactly once after the fold. // Scenario: steer once at step 1 (6k chars), then again at step 2 (12k) - // so the fold's cut can reach PAST the first steering event. Unpinned, - // the fold would swallow the first steer and the request that goes out - // would carry chars nothing measured. + // so the fold's cut can reach PAST the first steering event. Unpinned, the + // fold would swallow the first steer. const fixture = buildReactiveFixture({ script: ['tool', 'tool', 'done'], contextWindow: 2_000, @@ -1827,44 +1822,72 @@ describe('reactive overflow recovery in the streaming backend', () => { assert.equal(fixture.events.filter((event) => event.type === 'steering_message').length, 2); }); - test('the capacity verdict measures the steering payload the provider will actually receive', async () => { - // The base request (priors + one tool step) fits the window; the injected - // steering alone pushes the REAL request over it. Steering joins the - // request BEFORE shaping, so the single final-request verdict measures - // the payload the provider will receive and rescues it (one capacity - // fold) instead of silently sending an over-window request. The old - // order — append after the verdict — made the verdict blind to steering: - // no fold, no exhaustion, an unmeasured over-window request. - const bulkSteer = `CAPACITY_STEER_SENTINEL ${'S'.repeat(20_000)}`; + for (const scenario of [ + { name: 'without a declaration', withoutContextWindow: true, declareContextWindow: false }, + { + name: 'below a declared provider window', + contextWindow: 200_000, + declareContextWindow: true, + }, + ]) { + test(`suggests a Maka window after provider overflow ${scenario.name}`, async () => { + // The one fold is spent on the first rejection and the resend is + // rejected again: the turn surfaces the error, and only then does the + // note offer the last accepted total (120, before any fold) as a + // window. A send that overflows once, folds and completes says nothing. + const fixture = buildReactiveFixture({ + script: ['tool', 'overflow', 'overflow'], + ...scenario, + }); + await runTurn(fixture); + assert.equal(complete(fixture)?.stopReason, 'error'); + + const note = fixture.messages.find( + (message): message is { type: 'system_note'; kind: string; data?: unknown } => + (message as { type?: string }).type === 'system_note', + ); + assert.equal(note?.kind, 'context_window_suggestion'); + assert.deepEqual(note?.data, { + suggestedContextWindow: 120, + ...(scenario.declareContextWindow ? { declaredContextWindow: 200_000 } : {}), + }); + }); + } + + test('a send that overflows once, folds and completes suggests nothing', async () => { const fixture = buildReactiveFixture({ - script: ['tool', 'done'], - contextWindow: 5_000, - bigPriors: true, + script: ['tool', 'overflow', 'done'], + withoutContextWindow: true, }); - let pullCount = 0; - await runTurn(fixture, 'immediate', () => { - pullCount += 1; - // Steer at the SECOND step boundary, after the tool step: the verdict - // owner (step >= 1) must see the grown payload, not the step-0 baseline. - return pullCount === 2 - ? [{ id: 'lease-bulk', messageId: 'message-bulk', content: { text: bulkSteer } }] - : []; + await runTurn(fixture); + assert.equal(complete(fixture)?.stopReason, 'end_turn'); + // The fold itself is noted (context_compacted); the window suggestion is not. + assert.equal( + fixture.messages.some( + (message) => + (message as { type?: string; kind?: string }).type === 'system_note' && + (message as { kind?: string }).kind === 'context_window_suggestion', + ), + false, + ); + }); + + test('does not suggest a smaller Maka window when the declaration was already crossed', async () => { + const fixture = buildReactiveFixture({ + script: ['tool', 'overflow', 'overflow'], + contextWindow: 100, + declareContextWindow: true, }); + await runTurn(fixture); - const outcome = complete(fixture); - if (outcome?.stopReason === 'end_turn') { - // Rescued: the capacity owner reacted to the steering-inclusive payload - // with a mid-turn fold, and the delivered request still carries the - // steering exactly once. - assert.equal(fixture.recorded.length >= 1 || fixture.summarizerCalls() >= 1, true); - const finalPrompt = JSON.stringify(fixture.model.doStreamCalls.at(-1)?.prompt); - assert.equal(countOccurrences(finalPrompt, 'CAPACITY_STEER_SENTINEL'), 1); - } else { - // Not rescuable: the verdict terminates explicitly instead of sending - // an unmeasured over-window request. - assert.equal(outcome?.stopReason, 'error'); - } - assert.equal(fixture.events.filter((event) => event.type === 'steering_message').length, 1); + assert.equal( + fixture.messages.some( + (message) => + (message as { type?: string; kind?: string }).type === 'system_note' && + (message as { kind?: string }).kind === 'context_window_suggestion', + ), + false, + ); }); }); diff --git a/packages/runtime/src/__tests__/provider-conformance.test.ts b/packages/runtime/src/__tests__/provider-conformance.test.ts index 9be9efdafd..3cdd0bcb1a 100644 --- a/packages/runtime/src/__tests__/provider-conformance.test.ts +++ b/packages/runtime/src/__tests__/provider-conformance.test.ts @@ -25,6 +25,7 @@ import { anthropic } from '@ai-sdk/anthropic'; import { generateText, isStepCount, streamText, tool, type ModelMessage } from 'ai'; import { z } from 'zod'; import { fetchProviderModels } from '../model-fetcher.js'; +import { resetStreamUsageFallbackMemory } from '../stream-usage-fallback-fetch.js'; import { buildProviderOptions, getAIModel } from '../model-factory.js'; import { resolveOAuthSubscriptionAccessToken } from '../subscription-credentials.js'; import { testConnection } from '../test-connection.js'; @@ -1071,6 +1072,56 @@ describe('models.dev provider conformance', () => { assert.equal(probedPath, '/v1/responses'); }); + for (const [label, providerType] of [ + ['a plain OpenAI-compatible relay', 'openai-compatible'], + ['local Ollama', 'ollama'], + ] as const) { + test(`${label} requests usage in streamed chat completions by default`, async () => { + // Usage is the only signal the runtime's context handling reads (#4559). + // A Chat Completions server returns none unless asked, so every + // OpenAI-compatible adapter asks unless the registry opts it out. + let requestBody: Record | undefined; + const server = await startJsonServer(async (request, response) => { + assert.equal(request.method, 'POST'); + assert.equal(request.url, '/v1/chat/completions'); + requestBody = JSON.parse(await readBody(request)) as Record; + respondOpenAIStream(response, [ + { + id: 'chatcmpl-compatible-stream', + object: 'chat.completion.chunk', + created: 1, + model: 'relay-model', + choices: [ + { index: 0, delta: { role: 'assistant', content: 'ok' }, finish_reason: 'stop' }, + ], + usage: { prompt_tokens: 21, completion_tokens: 1, total_tokens: 22 }, + }, + ]); + }); + const connection: LlmConnection = { + slug: providerType, + name: label, + providerType, + baseUrl: `${server.url}/v1`, + defaultModel: 'relay-model', + enabled: true, + createdAt: 1, + updatedAt: 1, + }; + + const result = streamText({ + model: getAIModel({ connection, apiKey: 'test-key', modelId: 'relay-model' }), + prompt: 'Reply ok.', + }); + + assert.equal(await result.text, 'ok'); + assert.deepEqual(requestBody?.stream_options, { include_usage: true }); + const usage = await result.usage; + assert.equal(usage.inputTokens, 21); + assert.equal(usage.outputTokens, 1); + }); + } + test('Ollama Cloud requests usage in streamed chat completions', async () => { let requestBody: Record | undefined; const server = await startJsonServer(async (request, response) => { @@ -1114,6 +1165,72 @@ describe('models.dev provider conformance', () => { assert.equal(usage.totalTokens, 9); }); + test('a relay that rejects stream_options is answered once without the field', async () => { + // Asking for usage is the default, but a strict relay that rejects unknown + // fields would otherwise 400 every streaming request with no user-facing + // way to switch the ask off. One retreat, remembered for the endpoint: the + // request goes out again without `stream_options`, and the connection then + // simply reports no usage (#4559). + resetStreamUsageFallbackMemory(); + const bodies: Record[] = []; + const server = await startJsonServer(async (request, response) => { + const body = JSON.parse(await readBody(request)) as Record; + bodies.push(body); + if ('stream_options' in body) { + respondJson(response, 400, { + error: { + message: 'Unrecognized request argument supplied: stream_options', + type: 'invalid_request_error', + param: 'stream_options', + code: null, + }, + }); + return; + } + respondOpenAIStream(response, [ + { + id: 'chatcmpl-strict-relay', + object: 'chat.completion.chunk', + created: 1, + model: 'relay-model', + choices: [ + { index: 0, delta: { role: 'assistant', content: 'ok' }, finish_reason: 'stop' }, + ], + }, + ]); + }); + const connection: LlmConnection = { + slug: 'strict-relay', + name: 'Strict relay', + providerType: 'openai-compatible', + baseUrl: `${server.url}/v1`, + defaultModel: 'relay-model', + enabled: true, + createdAt: 1, + updatedAt: 1, + }; + + const result = streamText({ + model: getAIModel({ connection, apiKey: 'test-key', modelId: 'relay-model' }), + prompt: 'Reply ok.', + }); + + assert.equal(await result.text, 'ok'); + assert.equal(bodies.length, 2); + assert.deepEqual(bodies[0]?.stream_options, { include_usage: true }); + assert.equal('stream_options' in (bodies[1] ?? {}), false); + + // The endpoint is remembered: the next request never asks again. + const second = streamText({ + model: getAIModel({ connection, apiKey: 'test-key', modelId: 'relay-model' }), + prompt: 'Reply ok again.', + }); + assert.equal(await second.text, 'ok'); + assert.equal(bodies.length, 3); + assert.equal('stream_options' in (bodies[2] ?? {}), false); + resetStreamUsageFallbackMemory(); + }); + test('Hugging Face discovers tool-capable routed models and preserves its two-stage OpenAI wire', async () => { const discoveredModelId = 'openai/gpt-oss-120b'; const modelId = `${discoveredModelId}:preferred`; diff --git a/packages/runtime/src/__tests__/runtime-event-read-model.test.ts b/packages/runtime/src/__tests__/runtime-event-read-model.test.ts index 73eb22437a..9ffea826b8 100644 --- a/packages/runtime/src/__tests__/runtime-event-read-model.test.ts +++ b/packages/runtime/src/__tests__/runtime-event-read-model.test.ts @@ -2041,7 +2041,7 @@ describe('compareRuntimeReadModelMessages', () => { }); test('carries the cross-turn request anchor both ways and compares on it', () => { - const lastRequestAnchor = { inputTokens: 120, payloadChars: 48_000 }; + const lastRequestAnchor = { inputTokens: 120, outputTokens: 30 }; const anchored = ev({ id: 'evt-token-anchor', role: 'system', diff --git a/packages/runtime/src/__tests__/session-manager.test.ts b/packages/runtime/src/__tests__/session-manager.test.ts index 8b8c0e0a17..9abce3a7ae 100644 --- a/packages/runtime/src/__tests__/session-manager.test.ts +++ b/packages/runtime/src/__tests__/session-manager.test.ts @@ -3493,7 +3493,6 @@ describe('SessionManager manual compaction and quiescent session changes', () => now: nextNow(1), contextBudget: { name: 'manual-compact-accounting', - maxHistoryEstimatedTokens: 10_000, charsPerToken: 1, }, summarizeHistoryCompact: buildLlmHistorySummarizer({ diff --git a/packages/runtime/src/ai-sdk-backend.ts b/packages/runtime/src/ai-sdk-backend.ts index fe67a5fffb..5ccb33e29e 100644 --- a/packages/runtime/src/ai-sdk-backend.ts +++ b/packages/runtime/src/ai-sdk-backend.ts @@ -266,10 +266,7 @@ import { shouldAppendContextCompactionFailedOpenNote, type ContextBudgetPolicy, } from './context-budget.js'; -import { - evaluateHistoryCompactCheckpointReplay, - isHistoryCompactContentEvent, -} from './history-compaction.js'; +import { isHistoryCompactContentEvent } from './history-compaction.js'; import { canContinueHistoryCompactCheckpointForModel, historyCompactCheckpointToModelMessage, @@ -1556,6 +1553,11 @@ export class AiSdkBackend implements AgentBackend { // result.usage.inputTokens is cumulative across steps and would produce // misleading >100% percentages, so the per-step value is captured here. let lastStepInputTokens: number | undefined; + /** Tool count of the request that produced `lastStepInputTokens`. */ + let lastStepActiveToolCount: number | undefined; + // Output tokens of the same step: with the input they are the baseline the + // next request is judged from (everything the model produced is re-sent). + let lastStepOutputTokens: number | undefined; let streamStatus: LlmCallRecord['status'] = 'success'; let streamErrorClass: string | undefined; let runtimeSteps = 0; @@ -1563,6 +1565,14 @@ export class AiSdkBackend implements AgentBackend { let contextBudgetForTelemetry: ContextBudgetDiagnostic | undefined; let contextCompactedNoteWritten = false; let contextCompactionFailedOpenNoteWritten = false; + let contextProviderDroppingNoteWritten = false; + let contextWindowOverrunNoteWritten = false; + let contextReportedWindowNoteWritten = false; + let contextWindowSuggestionNoteWritten = false; + // Request index (0-based) at which the active prune last rewrote the + // request. A step Maka pruned is not append-only, so usage may legitimately + // shrink. + let pruneAppliedAtStep: number | undefined; const trace = new RunTrace({ sessionId: this.sessionId, turnId, @@ -1958,6 +1968,7 @@ export class AiSdkBackend implements AgentBackend { turnId, activeToolResultPruneIncludesNewestStep, (patch) => { + pruneAppliedAtStep = runtimeSteps; activeToolResultPruneDiagnosticPatch = mergeActiveToolResultPruneDiagnosticPatches( activeToolResultPruneDiagnosticPatch, patch, @@ -1969,22 +1980,9 @@ export class AiSdkBackend implements AgentBackend { midTurnCapacityHook, activeToolResultPruneHook, ); - // The verdict owner wraps the WHOLE shaping pipeline: hooks shape, one - // owner measures the final payload and decides pass/terminate. - const requestProjection = - midTurnState && midTurnCapacityHook && shapedProjection - ? this.compaction.buildMidTurnFinalRequestRescue({ - shaped: shapedProjection, - reentry: composeRequestProjection( - undefined, - midTurnCapacityHook, - activeToolResultPruneHook, - )!, - state: midTurnState, - providerTools, - charsPerToken: this.input.contextBudget?.charsPerToken ?? 4, - }) - : shapedProjection; + // Hooks shape; nothing measures the final payload. Whether it fits is + // the provider's answer (#4559). + const requestProjection = shapedProjection; const completedProviderSteps: RequestProjectionContext['completedSteps'][number][] = []; let requestMessages: ModelMessage[] = messages; @@ -2166,9 +2164,153 @@ export class AiSdkBackend implements AgentBackend { const stepUsage = event.usage; providerStepUsage = stepUsage; if (!stepUsage) sawUnusableStepUsage = true; + // Silent eviction / rewrite check (#4559): this step only + // appended (no fold, no prune, no image omission) yet the + // provider counted no more input tokens than for the previous + // request. Not-greater, not strictly-fewer: a provider that + // truncates to a fixed window (Ollama's `num_ctx`) reports the + // same total on every later request while Maka keeps + // appending, so a plateau is the signal, and an equal count + // after an append is already impossible without provider-side + // eviction or rewriting. Input against input: the previous + // reply's reasoning may not be resent, so input + output is + // not the floor of the next input on every wire. + const completedRequestIndex = runtimeSteps - 1; + // A finalization step resolves an empty tool set, so its + // request legitimately drops several thousand schema tokens + // with no fold, prune or image omission. Maka shaped that + // request; the provider did not drop anything. + const toolSchemaShrank = + lastStepActiveToolCount !== undefined && + activeToolsForRequest.length < lastStepActiveToolCount; + if ( + !contextProviderDroppingNoteWritten && + !toolSchemaShrank && + midTurnState && + completedRequestIndex >= 1 && + lastStepInputTokens !== undefined && + midTurnState.replacedStepNumber !== completedRequestIndex && + pruneAppliedAtStep !== completedRequestIndex && + midTurnState.omittedImageToolResults.size === 0 && + stepUsage !== undefined && + Number.isFinite(stepUsage.inputTokens) && + stepUsage.inputTokens > 0 && + stepUsage.inputTokens <= lastStepInputTokens + ) { + contextProviderDroppingNoteWritten = true; + const note: SystemNoteMessage = { + type: 'system_note', + id: this.newId(), + turnId, + ts: this.now(), + kind: 'context_provider_dropping', + }; + await this.input.appendMessage(note).catch(() => {}); + } // Fail closed: reset on every step boundary so a missing final // step's usage does not leave a stale value from an earlier step. + // The reply needed more room than the declared window had + // left after this request's own input. Both halves are the + // provider's numbers, read after the fact: the reserve that + // should have kept them apart was measured from a smaller + // previous reply. Say so once per send; the next request + // folds anyway because the baseline now exceeds the window. + if ( + !contextWindowOverrunNoteWritten && + midTurnState?.capacity !== undefined && + stepUsage !== undefined && + Number.isFinite(stepUsage.inputTokens) && + stepUsage.inputTokens > 0 && + Number.isFinite(stepUsage.outputTokens) && + stepUsage.outputTokens > 0 && + stepUsage.inputTokens + stepUsage.outputTokens > midTurnState.capacity + ) { + contextWindowOverrunNoteWritten = true; + const note: SystemNoteMessage = { + type: 'system_note', + id: this.newId(), + turnId, + ts: this.now(), + kind: 'context_window_overrun', + data: { + usedTokens: stepUsage.inputTokens + stepUsage.outputTokens, + declaredContextWindow: midTurnState.capacity, + }, + }; + await this.input.appendMessage(note).catch(() => {}); + } + // Nothing declared, and the provider accepted a request past + // the window this model reports. Every other signal in this + // design stays dark there: no rejection to recover from, no + // plateau to read, and no declaration to arm the proactive + // threshold, so the session degrades quietly and + // indefinitely (#4634). Report the two real numbers and + // leave the decision with the user: a reported window is a + // hint, and Maka still declares nothing on their behalf. + // + // Once per crossing, not once per send. On these providers + // usage keeps growing past the line (305K → 322K observed), + // so the note fires on the transition: the previous accepted + // total was still inside the reported window and this one is + // not. The baseline carries that previous total across + // sessions through the persisted anchor, so a resumed + // session does not repeat a crossing it already reported. + if ( + !contextReportedWindowNoteWritten && + midTurnState !== undefined && + midTurnState.capacity === undefined && + stepUsage !== undefined && + Number.isFinite(stepUsage.inputTokens) && + stepUsage.inputTokens > 0 && + Number.isFinite(stepUsage.outputTokens) + ) { + const reported = resolveSelectedModelContextWindow( + this.input.connection, + this.input.modelId, + ); + const used = stepUsage.inputTokens + Math.max(0, stepUsage.outputTokens); + // `baselineTokens` still describes the request before this + // one: the capacity hook sets it from the previous step, or + // from the persisted anchor on a send's first request. + const previousTotal = midTurnState.baselineTokens; + const crossedNow = + reported !== undefined && + used > reported && + (previousTotal === undefined || previousTotal <= reported); + if (reported !== undefined && crossedNow) { + contextReportedWindowNoteWritten = true; + const note: SystemNoteMessage = { + type: 'system_note', + id: this.newId(), + turnId, + ts: this.now(), + kind: 'context_reported_window_exceeded', + data: { usedTokens: used, reportedContextWindow: reported }, + }; + await this.input.appendMessage(note).catch(() => {}); + } + } lastStepInputTokens = stepUsage?.inputTokens; + lastStepOutputTokens = stepUsage?.outputTokens; + lastStepActiveToolCount = activeToolsForRequest.length; + // The provider cut this reply at an output limit. Which one + // decides whether folding helps: a reply that stopped at the + // `maxOutputTokens` Maka itself sends was cut by Maka's own + // budget, and the next request carries the same budget, so + // folding history would shrink the context every step without + // touching the constraint. Only a cut below that budget is + // the provider running out of room, which no local number + // predicted; fold once before the next request (#4559). + if (midTurnState && event.finishReason === 'length') { + const outputLimit = midTurnState.modelOutputLimitTokens; + const replyTokens = stepUsage?.outputTokens; + const cutByOwnBudget = + outputLimit > 0 && + replyTokens !== undefined && + Number.isFinite(replyTokens) && + replyTokens >= outputLimit; + if (!cutByOwnBudget) midTurnState.pendingLengthFold = true; + } if (stepUsage) { completedStepUsage = mergeNormalizedUsage(completedStepUsage, stepUsage); this.cumulativeUsageCheckpoint = mergeNormalizedUsage( @@ -2409,9 +2551,7 @@ export class AiSdkBackend implements AgentBackend { turnId, stepNumber: runtimeSteps, currentMessages: attemptMessages, - providerTools, activeTools: activeToolsForRequest, - systemPromptChars: requestSystemPrompt?.length ?? 0, queue, onDiagnosticPatch: onMidTurnDiagnosticPatch, origin: scope, @@ -2445,6 +2585,37 @@ export class AiSdkBackend implements AgentBackend { attemptMessages = recoveredProjection?.messages ?? recovered.messages; continue; } + // Window suggestion (#4559): the provider rejected a request and + // no recovery is left — the one fold is spent, or there was no + // seam. The baseline is a proven-fit total (input + output of an + // accepted request), so it is a number the user can declare; the + // trigger is `>=`, so declaring exactly it folds before this + // point next time. Once per send, and only when the turn is + // about to surface the error rather than continue. + const acceptedTotal = midTurnState?.lastAcceptedTotalTokens; + if ( + !contextWindowSuggestionNoteWritten && + failure.kind === 'context_overflow' && + midTurnState && + acceptedTotal !== undefined && + (midTurnState.capacity === undefined || acceptedTotal < midTurnState.capacity) + ) { + contextWindowSuggestionNoteWritten = true; + const note: SystemNoteMessage = { + type: 'system_note', + id: this.newId(), + turnId, + ts: this.now(), + kind: 'context_window_suggestion', + data: { + suggestedContextWindow: acceptedTotal, + ...(midTurnState.capacity !== undefined + ? { declaredContextWindow: midTurnState.capacity } + : {}), + }, + }; + await this.input.appendMessage(note).catch(() => {}); + } const idleWatchdogRecovery = settledWatchdogTimeout?.phase === 'idle' && idleWatchdogRetryCount < MAX_IDLE_WATCHDOG_RETRIES_PER_STEP && @@ -2740,13 +2911,17 @@ export class AiSdkBackend implements AgentBackend { } return undefined; })(); - // The anchor the NEXT turn estimates its first request from — see + // The anchor the NEXT turn judges its first request from — see // `LastRequestAnchor`. `input` below is the sum across this send's - // steps and anchors nothing. Either half missing (no usable usage - // sample, or no mid-turn seam to measure the payload) drops the - // whole pair and the next turn cold starts. + // steps and anchors nothing; the LAST step's real input and output + // are what the next request re-sends. No usable input count, no + // anchor: the next turn then has no proactive fold until its first + // accepted request. const anchorInputTokens = finitePositive(lastStepInputTokens); - const anchorPayloadChars = finitePositive(midTurnState?.lastRequestPayloadChars); + const anchorOutputTokens = + lastStepOutputTokens !== undefined && Number.isFinite(lastStepOutputTokens) + ? Math.max(0, lastStepOutputTokens) + : undefined; // One shared usage payload for the durable message and the live // event: twin per-field literals drifted before (#4019), so a field // now has exactly one definition site. @@ -2775,11 +2950,13 @@ export class AiSdkBackend implements AgentBackend { ? { contextRemaining: contextRemainingForUsage } : {}), ...(providerRequestTraceId ? { providerRequestTraceId } : {}), - ...(anchorInputTokens !== undefined && anchorPayloadChars !== undefined + ...(anchorInputTokens !== undefined ? { lastRequestAnchor: { inputTokens: anchorInputTokens, - payloadChars: anchorPayloadChars, + ...(anchorOutputTokens !== undefined + ? { outputTokens: anchorOutputTokens } + : {}), }, } : {}), @@ -3400,81 +3577,9 @@ export class AiSdkBackend implements AgentBackend { ); } - const maxHistoryTokens = contextBudget?.maxHistoryEstimatedTokens; - // FALLBACK, not a second authority: step 0's anchored estimate measures the - // whole outgoing payload against the real window, where this gate only - // weighs prior history events at chars/4. It stands in only where that - // estimate cannot reach — no anchor, no mid-turn seam, or no declared - // window — so an oversized history never goes out unshaped. - const needsCompaction = - !( - midTurnState?.capacity !== undefined && midTurnState.lastRequestInputTokens !== undefined - ) && - maxHistoryTokens !== undefined && - estimateRuntimeEventsTokens(runtimeContext, contextBudget?.charsPerToken) > maxHistoryTokens; - if ( - needsCompaction && - contextBudget?.historyCompact?.enabled === true && - this.compaction.hasHistoryCompactCheckpointWriter() - ) { - const automaticMemoryDecision = automaticMemory - ? this.automaticMemoryCompactionDecision() - : undefined; - const automaticMemorySource = automaticMemoryDecision - ? lastNonCompactRuntimeEvent(priorRuntimeContext) - : undefined; - const compactResult = await this.compaction.compactHistory( - { - turnId: input.turnId, - runId: scope.runId, - runtimeContext: priorRuntimeContext, - runtimeContextRunHeaders: input.runtimeContextRunHeaders, - }, - automaticMemorySource - ? { - runId: automaticMemorySource.runId, - turnId: automaticMemorySource.turnId, - runtimeEventId: automaticMemorySource.id, - disposition: automaticMemoryDecision!.disposition, - } - : undefined, - ); - let durableCheckpoint = compactResult.checkpoint; - if (!durableCheckpoint && compactResult.outcome.kind === 'unchanged') { - try { - durableCheckpoint = await Promise.resolve(this.input.loadHistoryCompactCheckpoint?.()); - } catch { - durableCheckpoint = undefined; - } - } - if (durableCheckpoint) { - const replay = buildHistoryCompactCheckpointFailOpenContext( - durableCheckpoint, - priorRuntimeContext, - contextBudget!, - priorRuntimeContext, - ); - runtimeContext = replay.events; - projectedHistoryCompactCheckpoint = replay.checkpoint; - if ( - replay.checkpoint && - compactResult.outcome.kind === 'compacted' && - automaticMemoryDecision?.dispatch && - automaticMemory - ) { - this.dispatchAutomaticMemoryCompaction(scope, { - checkpoint: replay.checkpoint, - activeTools: [], - }); - } - } - contextBudgetDiagnostic = mergeContextBudgetDiagnostic( - contextBudgetDiagnostic ?? - buildContextBudgetDiagnosticShell(priorRuntimeContext, runtimeContext, contextBudget), - compactResult.contextBudget ?? {}, - ); - } - + // No pre-turn estimate gate: the turn's first request is judged by the + // request-projection hook from the previous request's real usage, and by + // the provider when it goes out (#4559). // The boundary belongs to the runtime-event projection above. A gate that // falls back to the stored-message projection returns a prompt no // checkpoint shaped, so it reports none rather than one the request never @@ -4855,42 +4960,17 @@ function buildHistoryCompactCheckpointFailOpenContext( byTurn.set(event.turnId, [event]); } } - const maxTokens = policy.maxHistoryEstimatedTokens ?? Number.POSITIVE_INFINITY; - const replayPrefix = projectHistoryCompactCheckpointReplay( - checkpoint, - match.coveredRuntimeEvents, - [], - ); - let selectedTokens = - checkpoint.version === 3 - ? checkpoint.estimatedTokens - : estimateRuntimeEventsTokens(replayPrefix, charsPerToken); - const selectedGroups: RuntimeEvent[][] = []; - for (let index = turnOrder.length - 1; index >= 0; index -= 1) { - const group = byTurn.get(turnOrder[index]!) ?? []; - const groupTokens = estimateRuntimeEventsTokens(group, charsPerToken); - if (selectedTokens + groupTokens > maxTokens) break; - selectedGroups.unshift(group); - selectedTokens += groupTokens; - } + // Replay is structural: the checkpoint plus everything after its boundary. + // No size-based selection stands between them and dispatch; whether the + // result fits is the provider's answer (#4559). + const selectedGroups: RuntimeEvent[][] = turnOrder.map((turnId) => byTurn.get(turnId) ?? []); const replayTail = selectedGroups.flat(); const replayEvents = projectHistoryCompactCheckpointReplay( checkpoint, match.coveredRuntimeEvents, replayTail, ); - const replayTailForFit = checkpoint.version === 3 ? replayEvents : replayEvents.slice(1); - return evaluateHistoryCompactCheckpointReplay( - checkpoint, - replayTailForFit, - policy?.charsPerToken, - policy?.maxHistoryEstimatedTokens, - { - sourceReplayEvents: [...match.coveredRuntimeEvents, ...replayTail], - }, - ).fits - ? { events: replayEvents, checkpoint } - : { events: [...retainedCandidates] }; + return { events: replayEvents, checkpoint }; } function projectMemoryConversationPrefix( diff --git a/packages/runtime/src/ai-sdk-compaction-contract.ts b/packages/runtime/src/ai-sdk-compaction-contract.ts index 1bdd348587..698d7de64e 100644 --- a/packages/runtime/src/ai-sdk-compaction-contract.ts +++ b/packages/runtime/src/ai-sdk-compaction-contract.ts @@ -33,6 +33,13 @@ import type { import type { ModelFactory } from './model-adapter.js'; import type { ToolResultArchiveCapability } from './tool-result-archive-capability.js'; +/** + * Default output cap for a compaction summary. Measured summaries land around + * 1K tokens; the cap exists so a runaway summarizer cannot occupy the context + * it was asked to free, not to shape the summary (#4559). + */ +export const DEFAULT_HISTORY_COMPACT_MAX_OUTPUT_TOKENS = 8_000; + export interface HistoryCompactSummaryInput { sessionId: string; turnId: string; @@ -45,14 +52,14 @@ export interface HistoryCompactSummaryInput { previousCheckpoint?: HistoryCompactCheckpoint; newlyFoldedRuntimeEvents?: RuntimeEvent[]; /** - * Estimated provider-input ceiling for this compaction call. A compactor - * should fail before dispatch when its projection cannot fit this budget. - * The ceiling is absent when the selected model declares no context window. + * Output cap for the summary call. A summary is re-sent on every later + * request, so it is bounded outright rather than estimated: the summarizer's + * provider truncates at this cap (`finishReason: length`), and the compactor + * then asks once for a shorter one. Whether the compaction INPUT fits the + * summarizer's window is that provider's answer (`input_too_large`), never a + * local estimate's (#4559). */ - inputBudget?: { - maxEstimatedTokens?: number; - charsPerToken: number; - }; + maxOutputTokens?: number; abortSignal?: AbortSignal; /** * Physical-call tracking for this summarization, built by the backend (#1679). diff --git a/packages/runtime/src/ai-sdk-compaction.ts b/packages/runtime/src/ai-sdk-compaction.ts index 3138c9b6b9..c52724c6c7 100644 --- a/packages/runtime/src/ai-sdk-compaction.ts +++ b/packages/runtime/src/ai-sdk-compaction.ts @@ -50,10 +50,7 @@ import { mergeContextBudgetDiagnostic, type ContextBudgetPolicy, } from './context-budget.js'; -import { - evaluateHistoryCompactCheckpointReplay, - isHistoryCompactContentEvent, -} from './history-compaction.js'; +import { isHistoryCompactContentEvent } from './history-compaction.js'; import { canContinueHistoryCompactCheckpointForModel, canReplayHistoryCompactCheckpointForModel, @@ -69,7 +66,7 @@ import { type MalformedHistoryCompactSummaryReason, } from './history-compact-error.js'; import { createHash } from 'node:crypto'; -import type { ModelMessage } from './model-protocol.js'; +import type { ModelMessage, NormalizedUsage } from './model-protocol.js'; import type { ModelAdapter } from './model-adapter.js'; import type { RequestProjection, @@ -108,13 +105,10 @@ import { import { toolSchemaCharsForDiagnostics } from './request-shape.js'; import type { ModelCallAttempt, ModelCallKind } from '@maka/core/model-call-attempt'; import type { ProviderRequestTracker } from './provider-request-telemetry.js'; -import { - estimateNextRequestTokens, - exceedsHighWater, - planHistoryCompaction, -} from './history-compaction.js'; -import { resolveSelectedModelContextWindow } from './context-budget-policy.js'; -import { MATERIALIZED_IMAGE_TOKENS } from '@maka/core/attachments'; +import { planHistoryCompaction } from './history-compaction.js'; +import { resolveDeclaredContextWindow } from './context-budget-policy.js'; +import { lookupModelMetadata } from '@maka/core/model-metadata'; +import type { RuntimeExecutionConnection } from '@maka/core/llm-connections'; import { collectHistoricalImageToolResults, type HistoricalImageToolResult, @@ -311,10 +305,6 @@ export class AiSdkCompaction { } const charsPerToken = policy.charsPerToken ?? 4; - const estimatedTokensBefore = Math.max( - 1, - estimateRuntimeEventsTokens(runtimeContext, charsPerToken), - ); let previousCheckpoint: HistoryCompactCheckpoint | undefined; try { const loaded = await Promise.resolve(this.input.loadHistoryCompactCheckpoint?.()); @@ -336,14 +326,7 @@ export class AiSdkCompaction { if (previousCheckpoint) { const match = matchHistoryCompactCheckpointPrefix(previousCheckpoint, runtimeContext); if (!match.reason && match.successorRuntimeEvents.length === 0) { - const fit = evaluateHistoryCompactCheckpointReplay( - previousCheckpoint, - [], - charsPerToken, - policy.maxHistoryEstimatedTokens, - { sourceReplayEvents: runtimeContext }, - ); - if (fit.fits) { + { const projectedEvents = projectHistoryCompactCheckpointReplay( previousCheckpoint, match.coveredRuntimeEvents, @@ -404,10 +387,6 @@ export class AiSdkCompaction { }, newlyFoldedRuntimeEvents: [...newlyFoldedRuntimeEvents], ...(previousCheckpoint ? { previousCheckpoint } : {}), - inputBudget: { - maxEstimatedTokens: policy.maxHistoryEstimatedTokens ?? estimatedTokensBefore, - charsPerToken, - }, abortSignal: historyCompactAbortController.signal, ...(tracker ? { providerRequestTracker: tracker } : {}), }), @@ -435,29 +414,6 @@ export class AiSdkCompaction { }; } - const replayFit = evaluateHistoryCompactCheckpointReplay( - plan.checkpoint, - plan.tailRuntimeEvents, - charsPerToken, - policy.maxHistoryEstimatedTokens, - { sourceReplayEvents: runtimeContext }, - ); - if (!replayFit.fits) { - return { - outcome: { kind: 'failed', reason: replayFit.reason }, - contextBudget: mergeContextBudgetDiagnostic(diagnosticShell(runtimeContext), { - ...compactionDecisionDiagnosticPatch({ - stage: 'priorReplay', - sourceKind: 'runtimeEvents', - decision: 'failedOpen', - phase: 'pre_turn', - boundaryKind: 'historyCompact', - failOpenReason: replayFit.reason, - }), - }), - }; - } - try { await Promise.resolve(recorder(plan.checkpoint, input.turnId)); } catch { @@ -531,7 +487,7 @@ export class AiSdkCompaction { modelId: this.input.modelId, historyCompactRoute: this.input.historyCompactRoute, contextBudget: this.input.contextBudget, - inputBudget: input.inputBudget, + maxOutputTokens: input.maxOutputTokens, previousCheckpoint: input.previousCheckpoint, currentRunEventIds: input.runId ? input.source.foldedRuntimeEvents @@ -843,11 +799,12 @@ export class AiSdkCompaction { headAnchor, priorContentEvents, input.runtimeContextRunHeaders ?? [], - resolveSelectedModelContextWindow(this.input.connection, this.input.modelId), + resolveDeclaredContextWindow(this.input.connection, this.input.modelId), ); // Seed the turn's FIRST request with the last request the provider - // actually counted, so step 0 is estimated like every later step instead - // of guessing the whole payload at char/4. + // actually counted, so step 0 is judged by the same real number as every + // later step. No anchor means no proactive fold on step 0; the provider + // decides, and its rejection is recovered from. const persisted = persistedRequestAnchor( input.runtimeContext ?? [], state.priorRunHeaders, @@ -855,25 +812,31 @@ export class AiSdkCompaction { this.targetConnectionId, ); if (persisted) { - state.lastRequestInputTokens = persisted.inputTokens; - state.lastRequestPayloadChars = persisted.payloadChars; + state.baselineTokens = persisted.inputTokens + (persisted.outputTokens ?? 0); + state.lastAcceptedTotalTokens = state.baselineTokens; } + state.modelOutputLimitTokens = declaredModelOutputLimit( + this.input.connection, + this.input.modelId, + ); + if (persisted) state.replyReserveTokens = replyReserveTokens(persisted.outputTokens); return state; } /** - * Request-projection stage for the mid-turn capacity invariant: between - * steps of one turn, estimate the next provider request (last step's real - * usage + a signed char/4 payload delta, tool schemas included) against - * `contextWindow - reserve`; over the high-water, fold a safe completed - * prefix into a durable mid_turn checkpoint and continue the same turn on - * `[compact block, verbatim head anchor]`. + * Request-projection stage for proactive compaction: before each request of + * one turn, compare the baseline — the last provider-accepted request's real + * input plus real output tokens — against the context window the user + * declared, and over it fold a safe completed prefix into a durable + * checkpoint, continuing the turn on `[compact block, verbatim head anchor]`. * - * This hook never terminates the turn: every failure fails open with a - * diagnostic and the request goes out. The trigger threshold is approximate - * on purpose — a missed or spurious trigger costs at most one compaction, - * and whether the request actually fits is the provider's answer, not a - * local estimate's. + * Nothing here estimates whether a request fits: the baseline is the + * provider's own count, the window is the user's own number, and the part of + * the next request neither describes (tool results, user text, images + * appended since) is judged by the provider when the request goes out. This + * hook never terminates the turn: every failure fails open with a diagnostic + * and the request is sent; a rejection is recovered by one reactive fold + * (#4559). */ public buildMidTurnCapacityCompactProjection( turnId: string, @@ -887,11 +850,6 @@ export class AiSdkCompaction { abortSignal?: AbortSignal, ): RequestProjectionStage | undefined { if (!state) return undefined; - const policy = this.input.contextBudget!; - const compactPolicy = policy.historyCompact!; - const midTurn = compactPolicy.midTurn!; - const charsPerToken = policy.charsPerToken ?? 4; - const reserveTokens = midTurn.reserveTokens ?? 16_384; let acceptedProjection: AcceptedMidTurnCompactionProjection | undefined; return async (options) => { @@ -907,39 +865,21 @@ export class AiSdkCompaction { ) ?? projectedMessages; const keepProjection = (): RequestProjection | undefined => projectedMessages ? { messages: projectedMessages } : undefined; - // Real usage for the last finished step, read synchronously from the - // SDK's own step results (the same numbers the finish-step chunk - // carries) — no coupling to how far the stream consumer has advanced. - // Baseline = the last request's INPUT tokens only (see the state field - // doc: the payload delta already carries the step's output). The - // adapter fails closed on missing token counts (undefined, #972), and a - // provider can still report a zero input outright — either way a - // non-positive input count is unusable for estimation, so clear the - // baseline and let the estimate fall back to the whole-payload cold - // start instead of "0 + delta". - // - // The usage anchor is only meaningful PAIRED with the payload baseline - // of the request it was reported for (`lastRequestPayloadChars`). A - // successful overflow recovery restructures the request and resets that - // baseline to undefined: the send-global steps view still carries the - // dead attempt's last usage, but anchoring on it against the rejected - // request's chars would under-estimate the retry by the whole previous - // step growth — so a missing baseline forces the whole-payload cold - // start, exactly like a missing usage sample. - // - // Before the first step finishes nothing in-send has overwritten the - // seeded pair, so leave it alone rather than clearing a coherent anchor. + // Baseline = the last accepted request's REAL input tokens plus its REAL + // output tokens, read synchronously from the SDK's own step results. + // Everything the model produced last step is re-sent as input this step, + // so both halves are already in the next request; the only part no + // number describes yet is what was appended from outside the model, + // and that part is the provider's to judge. A missing or non-positive + // input count is no baseline at all — unknown, never zero. if (options.completedSteps.length > 0) { - const lastStepInputTokens = options.completedSteps.at(-1)?.usage?.inputTokens; - state.lastRequestInputTokens = - state.lastRequestPayloadChars !== undefined && - lastStepInputTokens !== undefined && - Number.isFinite(lastStepInputTokens) && - lastStepInputTokens > 0 - ? lastStepInputTokens - : undefined; + const lastUsage = options.completedSteps.at(-1)?.usage; + state.baselineTokens = usageBaselineTokens(lastUsage); + if (state.baselineTokens !== undefined) { + state.lastAcceptedTotalTokens = state.baselineTokens; + state.replyReserveTokens = replyReserveTokens(lastUsage?.outputTokens); + } } - // The turn's first request folds as a pre_turn boundary, like the // reactive step-0 recovery; later steps fold mid_turn. const phase = options.stepNumber === 0 ? 'pre_turn' : 'mid_turn'; @@ -960,56 +900,25 @@ export class AiSdkCompaction { }); return keepProjection(); }; - // A shaping failure records the step so the rescue re-entry does not - // re-run a shaper that already attempted and failed this step. - const shapeFailure = (diagnosticReason: string): RequestProjection | undefined => { - state.lastShapeFailure = { stepNumber: options.stepNumber }; - return failOpen(diagnosticReason); - }; - - // Trigger estimate: the last request's input tokens plus a SIGNED char/4 delta of - // this step's payload (system prompt + projected messages + active tool - // schemas) against the previous request's measured payload. Measured synchronously from - // the SDK's own projection — no ledger dependency — so a same-turn - // `tool_search` schema expansion or a large tool result both count. This - // position measures BEFORE later shapers (prune) run, so it can - // over-trigger; that is the recoverable direction, and the verdict owner - // re-measures the post-shaping payload. - const measuredMessages = projectedMessages ?? incomingMessages; - // Price the request dispatch will actually build, not the pre-dispatch - // inputs: a finalization step adds prompt fragments and sends no tools. - const dispatch = options.resolveDispatch(options.activeTools); - const activeToolsForStep = dispatch.activeTools; - const payloadChars = midTurnRequestPayloadChars( - measuredMessages, - providerTools, - activeToolsForStep, - dispatch.systemPromptChars, - charsPerToken, - ); - const forcedEstimate = state.forcedTriggerEstimate; - state.forcedTriggerEstimate = undefined; - const anchored = requestEstimateAnchor(state, payloadChars); - // The turn's FIRST request only gets a trigger when a previous turn left - // a usable anchor. Without one the estimate is the whole payload at - // chars/4 — the same guess the pre_turn gate already spends, and far too - // crude to start a summarizer on. With one, step 0 is judged exactly like - // every later step. - if ( - options.stepNumber < 1 && - forcedEstimate === undefined && - anchored.priorUsageTokens === undefined - ) { + // Two triggers, both real signals: the baseline crossed the user's + // window, or the provider cut the previous reply at its output limit. + const lengthFold = state.pendingLengthFold; + state.pendingLengthFold = false; + // The next request is at least the baseline, and its reply needs room on + // top of it. The room is measured from the last reply the model actually + // wrote, not from the largest one it could write: on a model whose + // output limit is a large fraction of its window (k3-256k reports + // 131,072 against 262,144) reserving the limit would fold at half the + // declared window, while a reply twice the size of the last one is the + // margin the session's own behaviour supports (#4634). + const overWindow = + state.capacity !== undefined && + state.baselineTokens !== undefined && + state.baselineTokens + state.replyReserveTokens >= state.capacity; + if (!overWindow && !lengthFold) { return keepProjection(); } - const estimate = forcedEstimate ?? estimateNextRequestTokens({ ...anchored, charsPerToken }); - if ( - forcedEstimate === undefined && - (state.capacity === undefined || !exceedsHighWater(estimate, state.capacity, reserveTokens)) - ) { - return keepProjection(); - } - + const activeToolsForStep = options.resolveDispatch(options.activeTools).activeTools; // Fold a safe completed prefix of the durable turn ledger into a // replacement projection (validate → persist), shared with the reactive // overflow path. This stage maps the outcome to the request-projection contract: @@ -1021,17 +930,17 @@ export class AiSdkCompaction { state, queue, minFlushedSteps: options.stepNumber, - referencePayloadChars: payloadChars, - providerTools, activeToolsForStep, - systemPromptChars: dispatch.systemPromptChars, memoryCompactionDecision, onMemoryCompaction, abortSignal, }); if (outcome.decision === 'fail') { - return shapeFailure(outcome.diagnosticReason); + return failOpen(outcome.diagnosticReason); } + // The fold replaced the request; the baseline described the old one. + // The next accepted request is the first measurement of the new shape. + state.baselineTokens = undefined; acceptedProjection = { sourceSignatures: incomingMessages.map(modelMessageSignature), projectedMessages: outcome.replacementMessages, @@ -1067,28 +976,17 @@ export class AiSdkCompaction { origin: ProviderRequestOrigin; queue: AsyncEventQueue; minFlushedSteps: number; - referencePayloadChars: number; - providerTools: readonly MakaTool[]; activeToolsForStep: readonly string[]; - systemPromptChars: number; memoryCompactionDecision?: () => AutomaticMemoryCompactionDecision; onMemoryCompaction?: (input: AutomaticMemoryCompactionDispatch) => void; phase?: 'pre_turn' | 'mid_turn'; abortSignal?: AbortSignal; }): Promise { - const { - turnId, - state, - queue, - providerTools, - activeToolsForStep, - systemPromptChars, - abortSignal, - } = input; - if (state.malformedSummaryFailure) { + const { turnId, state, queue, activeToolsForStep, abortSignal } = input; + if (state.summarizerFailure) { return { decision: 'fail', - diagnosticReason: state.malformedSummaryFailure, + diagnosticReason: state.summarizerFailure, }; } const summarizer = this.input.summarizeHistoryCompact!; @@ -1105,9 +1003,7 @@ export class AiSdkCompaction { const loadTurnRuntimeEvents = this.input.loadTurnRuntimeEvents!; const policy = this.input.contextBudget!; const compactPolicy = policy.historyCompact!; - const midTurn = compactPolicy.midTurn!; const charsPerToken = policy.charsPerToken ?? 4; - const reserveTokens = midTurn.reserveTokens ?? 16_384; // Coverage pool = the durable run ledger, read through the injected // seam. Covered events are persisted by construction (no crash window @@ -1204,12 +1100,6 @@ export class AiSdkCompaction { }, ...(previousCheckpoint ? { previousCheckpoint } : {}), newlyFoldedRuntimeEvents: [...newlyFoldedRuntimeEvents], - inputBudget: { - ...(state.capacity !== undefined - ? { maxEstimatedTokens: Math.max(1, state.capacity - reserveTokens) } - : {}), - charsPerToken, - }, ...(abortSignal ? { abortSignal } : {}), ...(midTurnTracker ? { providerRequestTracker: midTurnTracker } : {}), }); @@ -1218,21 +1108,23 @@ export class AiSdkCompaction { if (plan.decision === 'fail_open') { const diagnosticReason = plan.diagnosticReason ?? plan.reason; - if (isMalformedHistoryCompactSummaryReason(diagnosticReason)) { - state.malformedSummaryFailure = diagnosticReason; - } + // Latch every fail-open reason, not only the malformed ones. The baseline + // that fired this trigger survives a fail-open, so without the latch the + // next step evaluates the same condition and dispatches the same doomed + // summarizer call: a provider that answers slowly and fails (kimi's HTTP + // 200 with an error body) produced 15 such calls over 47 minutes before + // one main request (#4634). One attempt per send, then fail open (#4559). + state.summarizerFailure = diagnosticReason; return { decision: 'fail', diagnosticReason, }; } - // Lifecycle order is validate → persist → apply, where validate = - // materializable ∧ smaller ∧ replay-admissible. Replay applies the - // session's latest checkpoint BEFORE any high-water check, so a - // checkpoint that fails ANY of the three must never be persisted — it - // would poison every later projection even though this step correctly - // refused it. + // Lifecycle order is validate → persist → apply, where validate is + // materializable and replay-admissible. A checkpoint that fails either + // check must never be persisted because it would poison every later + // projection. const replayPlan = buildRuntimeEventModelReplayPlan(plan.replacementEvents, { toolActivityTurnIds: collectToolActivityTurnIds(orderedEvents), }); @@ -1258,40 +1150,9 @@ export class AiSdkCompaction { input.origin.runId, ), ); - // Apply the shape only when it actually shrinks the request versus the - // reference payload (the incoming request for the proactive hook, the - // request that overflowed for reactive recovery): a materialized - // replacement that is not smaller proves the summarizer's OUTPUT is - // unusable, reported as summarizer_failed via replacement_not_smaller. - const replacedPayloadChars = midTurnRequestPayloadChars( - replacementMessages, - providerTools, - activeToolsForStep, - systemPromptChars, - charsPerToken, - ); - if (replacedPayloadChars >= input.referencePayloadChars) { - return { - decision: 'fail', - diagnosticReason: 'replacement_not_smaller', - }; - } - // Replay admissibility uses the same complete-prefix capacity gate as - // recovery. Actual payload shrinkage was already checked above because - // only this owner can measure the fully materialized provider request. - const replayFit = evaluateHistoryCompactCheckpointReplay( - plan.checkpoint, - plan.checkpoint.version === 3 ? plan.replacementEvents : plan.replacementEvents.slice(1), - policy?.charsPerToken, - policy?.maxHistoryEstimatedTokens, - ); - if (!replayFit.fits) { - return { - decision: 'fail', - diagnosticReason: `replay_rejected_${replayFit.reason}`, - }; - } - + // Whether the replacement fits is the provider's answer once it goes out; + // no local measure stands between a materializable fold and dispatch + // (#4559). // The replacement is valid: durably persist the checkpoint BEFORE // applying the projection — the same order as the pre_turn path. A // persistence failure keeps raw messages and records write_failed. @@ -1343,9 +1204,7 @@ export class AiSdkCompaction { turnId: string; stepNumber: number; currentMessages: readonly ModelMessage[]; - providerTools: readonly MakaTool[]; activeTools: readonly string[]; - systemPromptChars: number; queue: AsyncEventQueue; onDiagnosticPatch: (patch: Partial) => void; origin: ProviderRequestOrigin; @@ -1366,28 +1225,10 @@ export class AiSdkCompaction { return image ? [[toolCallId, image] as const] : []; }), ); - state.lastRequestPayloadChars = undefined; - state.lastRequestInputTokens = undefined; + state.baselineTokens = undefined; return { messages: imageOmission.messages }; } - // The shrink baseline is the request the provider actually rejected. Its - // single owner is the verdict owner's per-request payload measure - // (state.lastRequestPayloadChars), recorded at the end of every - // request-projection run — the attempt-INITIAL messages undercount the rejected - // request by every same-turn tool step, and a baseline anchored there - // refuses folds that genuinely shrink the real request (review P1-1). - // The cold-start fallback only covers a send whose verdict owner never - // ran request projection (defensive; step 0 records the baseline too). - const referencePayloadChars = - state.lastRequestPayloadChars ?? - midTurnRequestPayloadChars( - input.currentMessages, - input.providerTools, - input.activeTools, - input.systemPromptChars, - this.input.contextBudget?.charsPerToken ?? 4, - ); const phase = input.stepNumber === 0 ? 'pre_turn' : 'mid_turn'; const outcome = await this.compactActiveRequestHistory({ turnId: input.turnId, @@ -1398,10 +1239,7 @@ export class AiSdkCompaction { // The stream has ended, so every completed step is already flushed; wait // only for the consumer to drain the durable ledger up to date. minFlushedSteps: state.flushedSteps, - referencePayloadChars, - providerTools: input.providerTools, activeToolsForStep: input.activeTools, - systemPromptChars: input.systemPromptChars, memoryCompactionDecision: input.memoryCompactionDecision, onMemoryCompaction: input.onMemoryCompaction, abortSignal: input.abortSignal, @@ -1436,113 +1274,10 @@ export class AiSdkCompaction { reason: 'overflow', }), ); - // A successful recovery restructures the request, so the rejected - // request's payload measure no longer describes what the retry sends. - // Reset the baseline: the capacity hook's usage anchor is only coherent - // paired with the payload chars of the SAME request, and a missing - // baseline forces the whole-payload cold-start estimate instead of a - // stale pairing against the dead attempt. The cross-turn seed goes with - // it: falling back to an even older request's anchor pairs worse, not - // better. - state.lastRequestPayloadChars = undefined; - state.lastRequestInputTokens = undefined; + // The fold replaced the request; the baseline described the rejected one. + state.baselineTokens = undefined; return { messages: outcome.replacementMessages }; } - - /** - * The single end-of-pipeline estimate owner for the mid-turn capacity - * invariant. Every request-projection stage only shapes; this wrapper measures the - * FINAL outgoing (messages, tools) payload — the bytes the provider will - * actually see, after capacity compaction, active tool-result pruning, and - * semantic/active-full compaction have all run — and spends the last - * chance to shrink it: - * - * - estimate = the last request's real INPUT tokens + signed char/4 delta - * against the previous request's measured payload (recorded here on - * every step, including step 0's baseline); the delta already carries - * the step's fresh output, so an output-inclusive baseline would count - * it twice, and an unusable usage sample falls back to the whole-payload - * cold start rather than a zero baseline; - * - over the window with no capacity attempt this step (the approximate - * trigger missed, e.g. growth the trigger under-weighted), force ONE - * capacity re-entry. - * - * Still over afterwards, the request goes out anyway: only the provider - * knows whether it fits, and a rejection is recovered from. - */ - public buildMidTurnFinalRequestRescue(input: { - shaped: RequestProjectionStage; - reentry: RequestProjectionStage; - state: MidTurnCapacityCompactState; - providerTools: readonly MakaTool[]; - charsPerToken: number; - }): RequestProjectionStage { - const { shaped, reentry, state, providerTools, charsPerToken } = input; - return async (options) => { - let result = await Promise.resolve(shaped(options)); - const omissionProjection = projectHistoricalImageOmissions( - result?.messages ?? options.messages, - state.omittedImageToolResults, - ); - if (omissionProjection) { - result = { ...(result ?? {}), messages: omissionProjection }; - } - const finalPayloadChars = (): number => { - // Measure the dispatched shape, so the payload recorded as the anchor's - // pair describes the same request the provider counts. - const dispatch = options.resolveDispatch(result?.activeTools ?? options.activeTools); - return midTurnRequestPayloadChars( - result?.messages ?? options.messages, - providerTools, - dispatch.activeTools, - dispatch.systemPromptChars, - charsPerToken, - ); - }; - let payloadChars = finalPayloadChars(); - // Same rule as the trigger: the turn's first request is measured only - // when a previous turn left a usable anchor to measure it against. - const anchoredAtStepZero = - requestEstimateAnchor(state, payloadChars).priorUsageTokens !== undefined; - if (state.capacity !== undefined && (options.stepNumber >= 1 || anchoredAtStepZero)) { - const estimateFinal = (): number => - estimateNextRequestTokens({ - ...requestEstimateAnchor(state, payloadChars), - charsPerToken, - }); - const estimate = estimateFinal(); - const capacityAttemptedThisStep = - state.replacedStepNumber === options.stepNumber || - state.lastShapeFailure?.stepNumber === options.stepNumber; - if (estimate > state.capacity && !capacityAttemptedThisStep) { - // One bounded capacity re-entry. Re-run only the capacity + prune - // shapers over the already-shaped projection; a second attempt - // after a same-step failure is pointless (the failure was not a - // trigger miss) and would double recorder counters and summarizer - // calls. - state.forcedTriggerEstimate = estimate; - const reshaped = await Promise.resolve( - reentry({ - ...options, - messages: result?.messages ?? options.messages, - ...(result?.activeTools ? { activeTools: result.activeTools } : {}), - }), - ); - state.forcedTriggerEstimate = undefined; - if (reshaped) { - result = { - ...(result ?? {}), - ...reshaped, - activeTools: reshaped.activeTools ?? result?.activeTools, - }; - } - payloadChars = finalPayloadChars(); - } - } - state.lastRequestPayloadChars = payloadChars; - return result; - }; - } } // -- moved helpers (defined in ai-sdk-backend, used only by cache write) ------- @@ -1664,28 +1399,40 @@ export function hasActiveToolResultPruneDiagnosticPatch( */ export class MidTurnCapacityCompactState { /** - * Raw serialized chars of the final provider request. Overflow recovery - * uses this as its shrink-reference baseline because it must compare the - * actual rejected projection with a candidate replacement. - * - * Seeded before the turn's first request from the anchor a previous turn - * persisted, so it can describe a request from an earlier send. + * The last provider-accepted request's real input tokens plus its real + * output tokens, as the provider counted them — everything already in the + * next request that a number describes. Seeded from the previous turn's + * persisted anchor, refreshed at every step boundary, and cleared by any + * fold (the checkpoint changed the request; the next accepted one is the + * first measurement). Undefined is "no baseline", never zero. */ - lastRequestPayloadChars: number | undefined; + baselineTokens: number | undefined; /** - * The last request's REAL input size: the inputTokens the provider reported - * for the last finished step. Never input+output — the signed payload delta - * already carries the step's freshly generated output (assistant text/tool - * calls) and its tool results, so an output-inclusive baseline would count - * them twice. Undefined when the last step's usage is missing or unusable - * (no positive input count); estimates then fall back to the whole-payload - * cold-start path — an unusable sample is unknown, never zero. - * - * Written and cleared as a pair with `lastRequestPayloadChars`, seeded from - * the previous turn's persisted anchor — see `LastRequestAnchor` for why the - * two only mean anything together. + * Input plus output of the last request the provider accepted, as it + * counted them. Unlike `baselineTokens` it survives a fold: it is not a + * trigger input but the number a rejected user can declare as their window, + * proven to fit because the provider already accepted it (#4559). */ - lastRequestInputTokens: number | undefined; + lastAcceptedTotalTokens: number | undefined; + /** + * Room the next reply may need: the model's declared output limit when the + * connection or metadata states one, else 0. A provider fact, never an + * estimate; it lets the trigger fire before a request that would otherwise + * be accepted but leave the reply no room (#4559). + */ + replyReserveTokens = 0; + /** + * The model's declared `maxOutputTokens`, as sent on every main request. + * A reply that stopped at this number was cut by Maka's own budget, not by + * the provider running out of context (#4559). + */ + modelOutputLimitTokens = 0; + /** + * Set when the provider cut the previous reply at its output limit + * (`finishReason: length`). The reply ran out of room, which no local + * number predicted; fold once before the next request. + */ + pendingLengthFold = false; /** Latest durable checkpoint (loaded or written) for roll-forward summaries. */ previousCheckpoint: HistoryCompactCheckpoint | undefined; /** Checkpoint accepted during this send; pins every later durable projection. */ @@ -1703,71 +1450,64 @@ export class MidTurnCapacityCompactState { * events enqueued at all. */ flushedSteps = 0; - /** - * Set by the final-request estimate owner to force one capacity re-entry on - * the current step, bypassing the (deliberately approximate) high-water - * trigger. Consumed by the capacity hook on its next invocation. - */ - forcedTriggerEstimate: number | undefined; /** Exact historical image results omitted after a provider overflow. */ omittedImageToolResults = new Map(); - /** - * The step of the capacity hook's most recent shaping failure. The rescue - * re-entry reads it so it never re-runs a shaper that already attempted and - * failed on the same step. - */ - lastShapeFailure: { stepNumber: number } | undefined; /** Malformed summaries spend one bounded repair budget for this whole Turn. */ - malformedSummaryFailure: MalformedHistoryCompactSummaryReason | undefined; + summarizerFailure: string | undefined; constructor( readonly headAnchor: RuntimeEvent, readonly priorContentEvents: readonly RuntimeEvent[], readonly priorRunHeaders: readonly AgentRunHeader[], - /** The model's declared context window, absent when it declares none. */ + /** + * The Maka window: the context window the USER declared for this model, + * a compaction target and nothing else. Absent when none is declared, + * and then no proactive fold ever runs — the provider decides (#4559). + */ readonly capacity: number | undefined, ) {} } /** - * Char measure of the FULL provider-visible request input: the system prompt - * (sent through the separate `system` field), the (projected) messages, and - * the serialized schemas of the active tool subset. Media is billed in tokens - * converted to chars, so the whole measure stays in one unit. The capacity - * trigger and the rescue re-entry both measure with this ONE function, so - * their comparisons against `lastRequestPayloadChars` are commensurable and - * same-turn tool-schema growth (a `tool_search` activation) is counted like - * any other payload growth. The system prompt is constant between adjacent - * requests — signed deltas cancel it — but the cold-start estimate (no usable - * usage sample) is the whole payload, so omitting it would under-estimate by - * exactly the system prompt. + * The model's declared output limit, from the connection's catalog entry or + * generated metadata — a provider fact the trigger may reserve for the next + * reply. 0 when nothing declares one. + */ +function declaredModelOutputLimit(connection: RuntimeExecutionConnection, modelId: string): number { + const limit = + connection.models?.find((model) => model.id === modelId)?.maxOutputTokens ?? + lookupModelMetadata(connection.providerType, modelId).maxOutputTokens; + return typeof limit === 'number' && Number.isFinite(limit) && limit > 0 ? limit : 0; +} + +/** + * The baseline one accepted request leaves for the next: its real input plus + * its real output tokens. Output is counted whole, reasoning included — a wire + * that does not resend reasoning makes this err high, and high is the safe + * direction for a trigger that can only ask for a compaction (#4559). + */ +/** + * Room to leave for the next reply, from the size of the last real one. * - * A media part is worth what the provider charges for it, not what it - * serializes to: materialized bytes reach the request as base64 or a byte map, - * where a 500 KB screenshot serializes to ~667K chars. Measuring that string - * makes one image look like a whole context window, which is how an affordable - * request became a terminal verdict (#4458). Billing it at the same constant - * the ledger's ruler uses keeps the two measures commensurable. + * Two real numbers and one bound: twice the last reply absorbs an answer that + * grows, and the cap keeps a single long reply from turning the reserve into + * the window. No previous reply means no reserve, never a guessed one. */ -function midTurnRequestPayloadChars( - messages: readonly ModelMessage[], - providerTools: readonly MakaTool[], - activeTools: readonly string[], - systemPromptChars: number, - charsPerToken: number, -): number { - let mediaParts = 0; - const serializedMessages = JSON.stringify(messages, (_key, value) => { - if (!isInlineImageFilePart(value)) return value; - mediaParts += 1; - return { type: value.type, mediaType: value.mediaType }; - }); - return ( - Math.max(0, Math.floor(systemPromptChars)) + - (serializedMessages?.length ?? 0) + - mediaParts * MATERIALIZED_IMAGE_TOKENS * Math.max(1, charsPerToken) + - toolSchemaCharsForDiagnostics(providerTools, activeTools) - ); +const MAX_REPLY_RESERVE_TOKENS = 8_000; + +function replyReserveTokens(lastReplyTokens: number | undefined): number { + if (lastReplyTokens === undefined || !Number.isFinite(lastReplyTokens) || lastReplyTokens <= 0) { + return 0; + } + return Math.min(lastReplyTokens * 2, MAX_REPLY_RESERVE_TOKENS); +} + +function usageBaselineTokens(usage: NormalizedUsage | undefined): number | undefined { + if (!usage) return undefined; + const input = usage.inputTokens; + if (!Number.isFinite(input) || input <= 0) return undefined; + const output = Number.isFinite(usage.outputTokens) ? Math.max(0, usage.outputTokens) : 0; + return input + output; } /** @@ -1801,25 +1541,6 @@ function persistedRequestAnchor( return undefined; } -/** - * The estimate inputs for the request about to go out: the paired anchor and - * the signed char delta against the payload that anchor was reported for. - * - * A delta wider than the whole payload means the pairing has already failed, so - * drop it and cold start: the whole payload against a zero anchor. - */ -function requestEstimateAnchor( - state: MidTurnCapacityCompactState, - payloadChars: number, -): { priorUsageTokens?: number; appendedChars: number } { - const anchor = state.lastRequestInputTokens; - const baseline = state.lastRequestPayloadChars; - if (anchor === undefined || baseline === undefined) return { appendedChars: payloadChars }; - const appendedChars = payloadChars - baseline; - if (Math.abs(appendedChars) > payloadChars) return { appendedChars: payloadChars }; - return { priorUsageTokens: anchor, appendedChars }; -} - /** * Outcome of folding the durable turn ledger into a replacement projection. * Shared by the proactive projection stage (which maps it to keepProjection / diff --git a/packages/runtime/src/context-budget-policy.ts b/packages/runtime/src/context-budget-policy.ts index bb7d5577e3..bf85b6a99f 100644 --- a/packages/runtime/src/context-budget-policy.ts +++ b/packages/runtime/src/context-budget-policy.ts @@ -19,7 +19,7 @@ import type { RuntimeExecutionConnection } from '@maka/core/llm-connections'; import { lookupModelMetadata } from '@maka/core/model-metadata'; -import { relayModelProfile } from '@maka/core/model-thinking'; +import { declaredContextWindow, relayModelProfile } from '@maka/core/model-thinking'; import type { ContextBudgetPolicy } from './context-budget.js'; export interface BuildDefaultContextBudgetPolicyOptions { @@ -27,24 +27,23 @@ export interface BuildDefaultContextBudgetPolicyOptions { modelId?: string; } +/** + * The shipped context-budget policy. It carries no history budget and no + * reserve: whether a request fits is the provider's answer, and the only + * proactive threshold is the context window the user declared for the model + * (see `resolveDeclaredContextWindow`), read by the compaction seam itself. + * What remains here are content policies — how one oversized Tool Result + * enters the request — and the compaction switches (#4559). + */ export function buildDefaultContextBudgetPolicy( - connection: RuntimeExecutionConnection, options: BuildDefaultContextBudgetPolicyOptions = {}, ): ContextBudgetPolicy { - const contextWindow = resolveSelectedModelContextWindow(connection, options.modelId); - const reserveTokens = defaultCompactReserveTokens(contextWindow); - const maxHistoryEstimatedTokens = defaultHistoryBudgetTokens( - connection, - contextWindow, - reserveTokens, - ); const surfaceName = (options.name ?? 'default-history-budget').replace( /-default-history-budget$/, '', ); return { name: options.name ?? 'default-history-budget', - ...(maxHistoryEstimatedTokens !== undefined ? { maxHistoryEstimatedTokens } : {}), staleToolResultPrune: { enabled: true, maxResultEstimatedTokens: 2_048, @@ -53,7 +52,7 @@ export function buildDefaultContextBudgetPolicy( historyCompact: { enabled: true, highWaterName: `${surfaceName}-history-compact`, - midTurn: { enabled: true, reserveTokens }, + midTurn: { enabled: true }, }, activeToolResultPrune: { enabled: true, @@ -64,29 +63,18 @@ export function buildDefaultContextBudgetPolicy( }; } -// Single owner of the compaction reserve default. The classic 16384 reserve -// assumed large-window models; on an 8K window it derived a 1-token history -// budget and a 1-token mid_turn high water — every multi-step turn ran the -// summarizer for a checkpoint the replay gate could never admit. The default -// is therefore bounded by the KNOWN window (a quarter of it, capped at 16384; -// peers bound the same way: opencode caps its buffer by the model's output -// limit, gemini-cli triggers at a window fraction). An unknown window keeps -// the classic constant. -function defaultCompactReserveTokens(contextWindow: number | undefined): number { - if (contextWindow === undefined) return 16_384; - return Math.min(16_384, Math.max(1, Math.floor(contextWindow / 4))); -} - -function defaultHistoryBudgetTokens( +/** + * The Maka window for the selected model: the context window the user declared, + * resolved by core's single owner of that rule (`declaredContextWindow`), or + * undefined when nothing is declared — and then no proactive compaction runs. + */ +export function resolveDeclaredContextWindow( connection: RuntimeExecutionConnection, - contextWindow: number | undefined, - reserveTokens: number, + modelId: string | undefined, ): number | undefined { - if (contextWindow !== undefined) { - return Math.max(1, contextWindow - reserveTokens); - } - if (connection.providerType === 'deepseek') return undefined; - return 32_000; + const selectedModelId = modelId ?? connection.defaultModel; + if (selectedModelId === undefined) return undefined; + return declaredContextWindow(connection, selectedModelId); } export function resolveSelectedModelContextWindow( diff --git a/packages/runtime/src/context-budget.ts b/packages/runtime/src/context-budget.ts index 1657d47f9d..b4eb92a7bd 100644 --- a/packages/runtime/src/context-budget.ts +++ b/packages/runtime/src/context-budget.ts @@ -49,12 +49,9 @@ import type { StaleToolResultPrunePolicy } from './tool-result-archive.js'; import { type ActiveToolResultPrunePolicy } from './active-tool-result-prune.js'; import { applyRuntimeEventHistoryCompact as applyRuntimeEventHistoryCompactNarrow, - evaluateHistoryCompactCheckpointReplay as evaluateHistoryCompactCheckpointReplayNarrow, isHistoryCompactContentEvent, type HistoryCompactionPolicy, - type HistoryCompactionReplayOptions, type HistoryCompactionReplayResult, - type HistoryCompactionCheckpointReplayFit, } from './history-compaction.js'; import type { RuntimeEvent } from '@maka/core/runtime-event'; @@ -68,11 +65,11 @@ import type { HistoryCompactCheckpoint } from './history-compact-checkpoint.js'; export interface ContextBudgetPolicy { name?: string; /** - * Approximate max model-visible prior-history tokens. This is an estimate - * used for shaping, not provider billing. + * Chars-per-token conversion for the CONTENT policies below (how large one + * Tool Result may be before it is archived) and for diagnostics. It takes + * part in no context-fit decision: whether a request fits is the provider's + * answer (#4559). Defaults to 4. */ - maxHistoryEstimatedTokens?: number; - /** Estimate conversion. Defaults to 4 chars/token, intentionally conservative for mixed text. */ charsPerToken?: number; /** Optional replay-only pruning for stale oversized tool results before whole-turn compaction. */ staleToolResultPrune?: StaleToolResultPrunePolicy; @@ -90,10 +87,11 @@ export interface BudgetedRuntimeContext { diagnostic: ContextBudgetDiagnostic; /** * The checkpoint this projection was actually replayed through — present only - * when it passed the prefix match and the replay fit, i.e. when these events - * really are `[block, tail]` rather than the raw prefix. + * when the prefix matched and these events really are `[block, tail]` rather + * than the raw prefix. Whether the resulting request fits is the provider's + * decision. * - * A loaded checkpoint that failed either gate is a checkpoint the caller + * A loaded checkpoint whose prefix does not match is a checkpoint the caller * holds and the projection ignored; the two must not be confused by anyone * reporting what a prompt was built from (#2323). */ @@ -107,9 +105,7 @@ export function applyRuntimeEventContextBudget( const prunePolicy = policy?.staleToolResultPrune; const pruneEnabled = prunePolicy?.enabled === true; const historyCompactEnabled = policy?.historyCompact?.enabled === true; - const enabled = Boolean( - policy?.maxHistoryEstimatedTokens || pruneEnabled || historyCompactEnabled, - ); + const enabled = pruneEnabled || historyCompactEnabled; if (!enabled) return undefined; if (!policy) return undefined; const charsPerToken = policy?.charsPerToken ?? 4; @@ -117,9 +113,7 @@ export function applyRuntimeEventContextBudget( const compacted = applyRuntimeEventHistoryCompactNarrow( events, policy?.historyCompact, - policy?.charsPerToken, - policy?.maxHistoryEstimatedTokens, - { charsPerToken }, + charsPerToken, ); // Stale Tool Result pruning is no longer a step of the budget: it is a // durable projection transition committed before this projection runs, and @@ -133,9 +127,6 @@ export function applyRuntimeEventContextBudget( const diagnostic: ContextBudgetDiagnostic = { enabled: true, ...(policy?.name ? { policyName: policy.name } : {}), - ...(policy.maxHistoryEstimatedTokens !== undefined - ? { maxHistoryEstimatedTokens: policy.maxHistoryEstimatedTokens } - : {}), estimatedTokensBefore, estimatedTokensAfter: estimateRuntimeEventsTokens(keptEvents, charsPerToken), keptTurns: keptTurnIds.size, @@ -191,9 +182,6 @@ export function buildContextBudgetDiagnosticShell( return { enabled: true, ...(policy?.name ? { policyName: policy.name } : {}), - ...(policy?.maxHistoryEstimatedTokens !== undefined - ? { maxHistoryEstimatedTokens: policy.maxHistoryEstimatedTokens } - : {}), estimatedTokensBefore: estimateRuntimeEventsTokens(before, charsPerToken), estimatedTokensAfter: estimateRuntimeEventsTokens(after, charsPerToken), keptTurns: turnCountAfter, @@ -227,30 +215,36 @@ export function mergeContextBudgetDiagnosticPatches( return mergeContextBudgetDiagnostic(left as ContextBudgetDiagnostic, right); } -export function shouldAppendContextCompactedNote( +// A history fold reaches the user as one note per send, whichever stage +// performed it: the replay of an existing checkpoint at turn start +// (`priorReplay`) or a fold the request-projection hook made before a request +// of this send (`activeStep`, pre_turn or mid_turn). Since #4486 every new fold +// happens in the hook, so a note keyed on replay alone would arrive one turn +// late — the turn that was compacted would show nothing (#4559). +function hasHistoryCompactDecision( contextBudget: ContextBudgetDiagnostic | undefined, + decision: 'replaced' | 'failedOpen', ): boolean { return ( contextBudget?.compactionDecisions?.some( - (decision) => - decision.stage === 'priorReplay' && - decision.boundaryKind === 'historyCompact' && - decision.decision === 'replaced', + (candidate) => + (candidate.stage === 'priorReplay' || candidate.stage === 'activeStep') && + candidate.boundaryKind === 'historyCompact' && + candidate.decision === decision, ) === true ); } +export function shouldAppendContextCompactedNote( + contextBudget: ContextBudgetDiagnostic | undefined, +): boolean { + return hasHistoryCompactDecision(contextBudget, 'replaced'); +} + export function shouldAppendContextCompactionFailedOpenNote( contextBudget: ContextBudgetDiagnostic | undefined, ): boolean { - return ( - contextBudget?.compactionDecisions?.some( - (decision) => - decision.stage === 'priorReplay' && - decision.boundaryKind === 'historyCompact' && - decision.decision === 'failedOpen', - ) === true - ); + return hasHistoryCompactDecision(contextBudget, 'failedOpen'); } export function minimalContextBudgetDiagnostic(): ContextBudgetDiagnostic { @@ -289,28 +283,10 @@ function mergeCompactionDecisionDiagnostics( export function applyRuntimeEventHistoryCompact( events: readonly RuntimeEvent[], policy: ContextBudgetPolicy | undefined, - options: HistoryCompactionReplayOptions = {}, ): HistoryCompactionReplayResult { return applyRuntimeEventHistoryCompactNarrow( events, policy?.historyCompact, policy?.charsPerToken, - policy?.maxHistoryEstimatedTokens, - options, - ); -} - -export function evaluateHistoryCompactCheckpointReplay( - checkpoint: HistoryCompactCheckpoint, - replayTail: readonly RuntimeEvent[], - policy: ContextBudgetPolicy | undefined, - options: HistoryCompactionReplayOptions = {}, -): HistoryCompactionCheckpointReplayFit { - return evaluateHistoryCompactCheckpointReplayNarrow( - checkpoint, - replayTail, - policy?.charsPerToken, - policy?.maxHistoryEstimatedTokens, - options, ); } diff --git a/packages/runtime/src/conversation-copy.ts b/packages/runtime/src/conversation-copy.ts index d35910590f..1b5d78bd02 100644 --- a/packages/runtime/src/conversation-copy.ts +++ b/packages/runtime/src/conversation-copy.ts @@ -901,15 +901,14 @@ function cloneAgentRunEvent( if (match.reason) return null; // Copy is an admission seam for the sectioned summary contract: a marked // checkpoint whose summary no longer satisfies the COMPLETE predicate — - // including the size floor, re-runnable here because the matched covered - // span is in hand — must not propagate into a fresh session. Unmarked - // legacy summaries stay copyable under the truncation-only load policy - // and keep their unmarked identity in the target. + // re-runnable here on structure and truncation (the size floor needs the + // summarizer call's usage, which a copy does not have) — must not + // propagate into a fresh session. Unmarked legacy summaries stay copyable + // under the truncation-only load policy and keep their unmarked identity + // in the target. if ( sourceCheckpoint.summaryFormat !== undefined && - findCheckpointSummaryDefect(sourceCheckpoint.summary, { - coveredRuntimeEvents: match.coveredRuntimeEvents, - }) !== undefined + findCheckpointSummaryDefect(sourceCheckpoint.summary) !== undefined ) { throw new Error(`Cannot copy invalid history compact checkpoint ${event.id}`); } diff --git a/packages/runtime/src/history-compact-checkpoint.ts b/packages/runtime/src/history-compact-checkpoint.ts index 21fc17ee00..8c3b29e264 100644 --- a/packages/runtime/src/history-compact-checkpoint.ts +++ b/packages/runtime/src/history-compact-checkpoint.ts @@ -217,16 +217,13 @@ export function buildHistoryCompactCheckpoint( throw new Error('History compact checkpoint requires valid provider compaction state'); } // The sectioned marker is proof that the complete predicate held, so only - // this builder assigns it — and only after re-checking against the covered - // span (structure, truncation, AND the size floor, since the covered events - // are in hand here at every construction seam, including copy). A caller - // with unvalidated free-form text must declare `legacy_freeform` instead of - // minting trust it did not earn. + // this builder assigns it — and only after re-checking structure and + // truncation at every construction seam, including copy. The size floor is + // the summarizer's own check: it needs the provider's usage for the call, + // which no construction seam has. A caller with unvalidated free-form text + // must declare `legacy_freeform` instead of minting trust it did not earn. if (!providerState && input.summaryFormat !== 'legacy_freeform') { - const defect = findCheckpointSummaryDefect(summary!, { - coveredRuntimeEvents: input.coveredRuntimeEvents, - ...(input.charsPerToken !== undefined ? { charsPerToken: input.charsPerToken } : {}), - }); + const defect = findCheckpointSummaryDefect(summary!); if (defect) { throw new Error(`History compact checkpoint summary failed validation: ${defect}`); } diff --git a/packages/runtime/src/history-compact-summarizer.ts b/packages/runtime/src/history-compact-summarizer.ts index bcfad2552c..a3d7cfd30f 100644 --- a/packages/runtime/src/history-compact-summarizer.ts +++ b/packages/runtime/src/history-compact-summarizer.ts @@ -24,14 +24,17 @@ import { SUMMARY_FORMAT_TEMPLATE, } from './history-compact-summary-validation.js'; import { effectiveReplayToolResultOutput } from './durable-tool-result-projection.js'; -import type { HistoryCompactSummaryInput } from './ai-sdk-compaction-contract.js'; +import { + DEFAULT_HISTORY_COMPACT_MAX_OUTPUT_TOKENS, + type HistoryCompactSummaryInput, +} from './ai-sdk-compaction-contract.js'; import { HistoryCompactSummarizerError, isMalformedHistoryCompactSummaryReason, } from './history-compact-error.js'; import { isTextHistoryCompactCheckpoint } from './history-compact-checkpoint.js'; -import { fitHistoryCompactMessages } from './history-compact-input-fit.js'; -import type { AiSdkUsageLike } from './model-adapter.js'; +import { normalizeAiSdkUsage, type AiSdkUsageLike } from './model-adapter.js'; +import { classifyError } from './provider-error-classification.js'; import { withProviderGenerateTracking } from './provider-request-telemetry.js'; export { HistoryCompactSummarizerError } from './history-compact-error.js'; @@ -78,6 +81,17 @@ const SUMMARIZATION_SYSTEM_PROMPT = [ 'Keep each section concise. Preserve exact file paths, function names, commands, and error messages.', ].join('\n'); +const SUMMARY_REQUEST_INSTRUCTION = + 'Now write the structured summary of the conversation above. Output only the summary.'; + +function shortenSummarizationSystemPrompt(): string { + return [ + SUMMARIZATION_SYSTEM_PROMPT, + '', + 'Your previous attempt was cut off at the output limit. Produce the same summary in well under half the length: keep every section, drop detail rather than sections.', + ].join('\n'); +} + function repairSummarizationSystemPrompt(reason: string): string { return [ SUMMARIZATION_SYSTEM_PROMPT, @@ -113,11 +127,20 @@ export function buildLlmHistorySummarizer(options: BuildLlmHistorySummarizerOpti ], }); } - const initialMessages = fitHistoryCompactMessages(projectedMessages, { - maxInputEstimatedTokens: input.inputBudget?.maxEstimatedTokens, - charsPerToken: input.inputBudget?.charsPerToken, - fixedInputChars: SUMMARIZATION_SYSTEM_PROMPT.length, + // The folded span usually ends on an assistant message. A chat-template + // model handed a conversation that already ends with its own turn emits + // an end-of-sequence token and nothing else (Ollama qwen2.5: finish + // `stop`, one output token, empty text), so the request must end with + // an instruction the model can answer. Hosted providers do not need the + // nudge and are not disturbed by it (#4559). + projectedMessages.push({ + role: 'user', + content: [{ type: 'text', text: SUMMARY_REQUEST_INSTRUCTION }], }); + // Nothing is trimmed on a local estimate: whether this input fits the + // summarizer's window is its provider's answer (`input_too_large`, which + // the planner retreats on), and the output is capped outright (#4559). + const maxOutputTokens = input.maxOutputTokens ?? DEFAULT_HISTORY_COMPACT_MAX_OUTPUT_TOKENS; // Handed over whole by the backend, which owns every input a tracker // needs — including the run, which no summarizer wiring can know (#1679). const providerRequestTracker = input.providerRequestTracker; @@ -133,34 +156,48 @@ export function buildLlmHistorySummarizer(options: BuildLlmHistorySummarizerOpti ...(input.abortSignal ? { abortSignal: input.abortSignal } : {}), }) : options.resolveModel(); - const generateSummary = async ( - step: number, - instructions: string, - messages: ModelMessage[], - ) => { + let step = 0; + const generateSummary = async (instructions: string, messages: ModelMessage[]) => { providerRequestTracker?.setStep(step); + step += 1; const result = await generateText({ model, instructions, messages, + maxOutputTokens, ...(options.providerOptions !== undefined ? { providerOptions: options.providerOptions } : {}), ...(input.abortSignal ? { abortSignal: input.abortSignal } : {}), }); - if (rawFinishReasonString(result.finishReason) === 'length') { - throw new HistoryCompactSummarizerError('output_length'); - } - const defect = findCheckpointSummaryDefect(result.text, { - coveredRuntimeEvents: input.source.foldedRuntimeEvents, - ...(input.inputBudget?.charsPerToken !== undefined - ? { charsPerToken: input.inputBudget.charsPerToken } - : {}), - }); - return { text: result.text, defect }; + const usage = normalizeAiSdkUsage(result.usage); + const truncated = rawFinishReasonString(result.finishReason) === 'length'; + // The size floor compares the summary with what it replaces. On a + // roll-forward the request carries the previous summary plus the new + // increment, so its input tokens are not the covered span and the + // floor would judge the wrong number; it applies to the initial fold + // only (#4559). + const spanUsage = + previousCheckpoint === undefined && usage !== undefined && usage.inputTokens > 0 + ? { inputTokens: usage.inputTokens, outputTokens: usage.outputTokens } + : undefined; + const defect = truncated + ? undefined + : findCheckpointSummaryDefect( + result.text, + spanUsage ? { summarizerUsage: spanUsage } : undefined, + ); + return { text: result.text, defect, truncated }; }; - const initial = await generateSummary(0, SUMMARIZATION_SYSTEM_PROMPT, initialMessages); + let initial = await generateSummary(SUMMARIZATION_SYSTEM_PROMPT, projectedMessages); + if (initial.truncated) { + // The provider cut the summary at the output cap. One shorter attempt; + // a second cut is the provider saying this span will not summarize + // inside the cap, and the fold fails open. + initial = await generateSummary(shortenSummarizationSystemPrompt(), projectedMessages); + if (initial.truncated) throw new HistoryCompactSummarizerError('output_length'); + } if (!initial.defect) return initial.text; if (!isMalformedHistoryCompactSummaryReason(initial.defect)) { throw new HistoryCompactSummarizerError(initial.defect); @@ -170,16 +207,19 @@ export function buildLlmHistorySummarizer(options: BuildLlmHistorySummarizerOpti // be bounded: one stricter attempt, then the caller's failure circuit // records the stable defect for this compaction input. const repairInstructions = repairSummarizationSystemPrompt(initial.defect); - const repairMessages = fitHistoryCompactMessages(projectedMessages, { - maxInputEstimatedTokens: input.inputBudget?.maxEstimatedTokens, - charsPerToken: input.inputBudget?.charsPerToken, - fixedInputChars: repairInstructions.length, - }); let repaired: Awaited>; try { - repaired = await generateSummary(1, repairInstructions, repairMessages); + repaired = await generateSummary(repairInstructions, projectedMessages); + if (repaired.truncated) throw new HistoryCompactSummarizerError('output_length'); } catch (error) { if (isAbortError(error)) throw error; + // The repair prompt is longer than the first one. If that is what pushed + // the fold past the summarizer provider's window, the rejection is the + // planner's retreat signal, not a repair failure to file under the + // initial defect (#4559). + if (classifyError(error) === 'ContextLength') { + throw new HistoryCompactSummarizerError('input_too_large', { cause: error }); + } throw new HistoryCompactSummarizerError(initial.defect, { cause: error instanceof HistoryCompactSummarizerError @@ -201,6 +241,12 @@ export function buildLlmHistorySummarizer(options: BuildLlmHistorySummarizerOpti } catch (error) { if (isAbortError(error)) throw error; if (error instanceof HistoryCompactSummarizerError) throw error; + // The summarizer's provider is the one judge of whether this fold fits + // its own window: a context-length rejection is the signal the planner + // retreats on, so it must keep its name here (#4559). + if (classifyError(error) === 'ContextLength') { + throw new HistoryCompactSummarizerError('input_too_large', { cause: error }); + } throw new HistoryCompactSummarizerError('provider_error', { cause: error }); } }; diff --git a/packages/runtime/src/history-compact-summary-validation.ts b/packages/runtime/src/history-compact-summary-validation.ts index 99a67442f9..5e4136c581 100644 --- a/packages/runtime/src/history-compact-summary-validation.ts +++ b/packages/runtime/src/history-compact-summary-validation.ts @@ -17,10 +17,7 @@ * under the License. */ -import type { RuntimeEvent } from '@maka/core/runtime-event'; import type { MalformedHistoryCompactSummaryReason } from './history-compact-error.js'; -import { estimateTokens } from './context-budget-helpers.js'; -import { estimateRuntimeEventsTokens } from './model-history.js'; // The single authority on what a history-compact checkpoint summary must look // like (#3029). The summarization prompt is built from the same constants and @@ -78,12 +75,13 @@ const TEMPLATE_PLACEHOLDER_LINES: ReadonlySet = new Set( ); // Floors for the incident's shape: folding a large span into a paragraph -// cannot be a faithful checkpoint. Folds above ~10k estimated tokens require -// at least ~200 estimated tokens of summary, both measured at the session's -// chars-per-token estimate so CJK-heavy sessions are floored consistently. -const LARGE_FOLD_ESTIMATED_TOKENS = 10_000; +// cannot be a faithful checkpoint. Folds whose summarizer request the provider +// counted above ~10k input tokens require at least ~200 output tokens of +// summary. Both sides are the summarizer call's REAL usage, so the floor holds +// in the provider's own tokenizer and needs no chars-per-token guess (#4559); +// a producer that reports no usage is not floored. +const LARGE_FOLD_INPUT_TOKENS = 10_000; const LARGE_FOLD_SUMMARY_TOKENS_FLOOR = 200; -const DEFAULT_CHARS_PER_TOKEN = 4; // Best-effort signals that a provider stopped mid-thought. Fence state is // derived by the structural scanner below, so write admission and legacy-load @@ -92,12 +90,14 @@ const DEFAULT_CHARS_PER_TOKEN = 4; const TRUNCATED_TAIL_PATTERN = /(?:\.{3}|[:：,，、;；…(（—])\s*$/u; export interface CheckpointSummaryFoldContext { - /** The FULL covered span the checkpoint replaces — not the newly folded - * increment, so rolling roll-forward compaction cannot slip a fragment - * past the size floor. */ - coveredRuntimeEvents: readonly RuntimeEvent[]; - /** The session's estimate; defaults to the shared 4-chars/token. */ - charsPerToken?: number; + /** + * The summarizer call's real usage for an INITIAL fold, as its provider + * counted it: input is the covered span, output is the summary. Absent when + * the producer reports none or the fold rolls a previous checkpoint + * forward (its input is then the increment, not the span); the size floor + * is not applied in either case. + */ + summarizerUsage?: { inputTokens: number; outputTokens: number }; } // #3029: a degraded provider completion — a 138-token free-form fragment @@ -119,19 +119,13 @@ export function findCheckpointSummaryDefect( } const truncationDefect = findTruncationDefect(trimmed, scan); if (truncationDefect !== undefined) return truncationDefect; - if (foldContext !== undefined) { - // Clamped so a zero/negative estimate cannot zero out the floor. - const charsPerToken = Math.max(1, foldContext.charsPerToken ?? DEFAULT_CHARS_PER_TOKEN); - // Both sides use the shared ceil-based estimator so the floor cannot - // disagree with the repository's estimated-token semantics at the - // boundary. - if ( - estimateTokens(trimmed.length, charsPerToken) < LARGE_FOLD_SUMMARY_TOKENS_FLOOR && - estimateRuntimeEventsTokens(foldContext.coveredRuntimeEvents, charsPerToken) > - LARGE_FOLD_ESTIMATED_TOKENS - ) { - return 'malformed_summary_too_small_for_fold'; - } + const usage = foldContext?.summarizerUsage; + if ( + usage !== undefined && + usage.outputTokens < LARGE_FOLD_SUMMARY_TOKENS_FLOOR && + usage.inputTokens > LARGE_FOLD_INPUT_TOKENS + ) { + return 'malformed_summary_too_small_for_fold'; } return undefined; } diff --git a/packages/runtime/src/history-compaction.ts b/packages/runtime/src/history-compaction.ts index 2c67346f55..ef4acfdd49 100644 --- a/packages/runtime/src/history-compaction.ts +++ b/packages/runtime/src/history-compaction.ts @@ -50,57 +50,10 @@ import { * provider, and a rejection is recovered from by compacting and retrying once. */ -export interface EstimateNextRequestTokensInput { - /** - * The last request's real INPUT tokens as reported by the provider — never - * input+output, because `appendedChars` is a delta against that request's - * payload and already carries the step's freshly generated output. - * Undefined on cold start or when the sample is unusable (no positive - * input count); the baseline is then zero and `appendedChars` carries the - * whole payload. - */ - priorUsageTokens?: number; - /** - * SIGNED char delta of the next request's payload versus the last measured - * request payload. Negative after compaction/pruning shrank the projection — - * the estimate must credit the shrink, or a compacted request would still be - * judged by the pre-compaction usage sample. - */ - appendedChars: number; - /** Estimate conversion; defaults to 4 chars/token. */ - charsPerToken?: number; -} - -/** - * Estimate the token size of the next provider request: the last request's real - * usage plus a signed char/4 payload delta for content the provider has not yet - * counted (or no longer carries). Without a usable usage sample the caller - * passes the whole payload as the delta against a zero baseline, so this stays - * one formula. This mirrors how surveyed peers avoid pure character guessing. - */ -export function estimateNextRequestTokens(input: EstimateNextRequestTokensInput): number { - const charsPerToken = Math.max(1, input.charsPerToken ?? 4); - const prior = - input.priorUsageTokens !== undefined && Number.isFinite(input.priorUsageTokens) - ? Math.max(0, Math.floor(input.priorUsageTokens)) - : 0; - return Math.max(0, prior + estimateSignedChars(input.appendedChars, charsPerToken)); -} - -/** Proactive threshold: the next request would cross `contextWindow - reserve`. */ -export function exceedsHighWater( - estimatedTokens: number, - contextWindow: number, - reserveTokens: number, -): boolean { - const highWater = Math.max(1, contextWindow - Math.max(0, reserveTokens)); - return estimatedTokens > highWater; -} - export interface SafePrefixOptions { /** Keep at least this many trailing events uncovered as the verbatim tail. */ reserveTailEvents?: number; - /** Retry a smaller prefix after a local summarizer input-fit rejection. */ + /** Retry a smaller prefix after the summarizer provider rejects its input. */ maxCoveredCount?: number; /** * Events that must stay in the verbatim tail: the boundary retreats to @@ -199,13 +152,6 @@ function straddlesToolPair(spans: readonly ToolPairSpan[], cut: number): boolean return false; } -function estimateSignedChars(chars: number | undefined, charsPerToken: number): number { - const value = Math.trunc(chars ?? 0); - if (!Number.isFinite(value) || value === 0) return 0; - const magnitude = Math.ceil(Math.abs(value) / charsPerToken); - return value > 0 ? magnitude : -magnitude; -} - // ============================================================================ // Orchestration: engine + checkpoint protocol + injected summarizer → decision // ============================================================================ @@ -335,7 +281,12 @@ export async function planHistoryCompaction( } catch (error) { if (error instanceof HistoryCompactSummarizerError) { if (error.reason === 'input_too_large') { - maxCoveredCount = boundary.coveredCount - 1; + // The summarizer's provider said this span does not fit its own + // window; that is the only fit signal the fold listens to. Retreat by + // half rather than by one event: each retreat is a real provider + // round trip, and the loop exits through no_safe_completed_span when + // even the smallest legal span is refused. + maxCoveredCount = Math.floor(boundary.coveredCount / 2); continue; } return { @@ -354,10 +305,8 @@ export async function planHistoryCompaction( // history. The default summarizer already threw with the same reasons; any // other producer is validated here. if (typeof compacted === 'string') { - const defect = findCheckpointSummaryDefect(compacted, { - coveredRuntimeEvents, - charsPerToken, - }); + // An external producer reports no usage; only the structural checks apply. + const defect = findCheckpointSummaryDefect(compacted); if (defect) { return { decision: 'fail_open', reason: 'summarizer_failed', diagnosticReason: defect }; } @@ -411,73 +360,20 @@ export interface HistoryCompactionPolicy { enabled: boolean; checkpoint?: HistoryCompactCheckpoint; highWaterName?: string; - midTurn?: { enabled: true; reserveTokens?: number }; + midTurn?: { enabled: true }; } -export interface HistoryCompactionReplayOptions { - charsPerToken?: number; - maxHistoryEstimatedTokens?: number; - sourceReplayEvents?: readonly RuntimeEvent[]; -} - -export type HistoryCompactionCheckpointReplayFit = - | { fits: true; checkpointTokens: number; replayTokens: number } - | { - fits: false; - checkpointTokens: number; - replayTokens: number; - reason: 'prefix_over_budget' | 'replacement_not_smaller'; - }; - export interface HistoryCompactionReplayResult { events: RuntimeEvent[]; checkpoint?: HistoryCompactCheckpoint; diagnosticPatch: Partial; } -/** The single current-policy gate for every checkpoint entering model replay. */ -export function evaluateHistoryCompactCheckpointReplay( - checkpoint: HistoryCompactCheckpoint, - replayTail: readonly RuntimeEvent[], - charsPerToken: number | undefined, - maxHistoryEstimatedTokens: number | undefined = undefined, - options: HistoryCompactionReplayOptions = {}, -): HistoryCompactionCheckpointReplayFit { - const charsPerTokenResolved = options.charsPerToken ?? charsPerToken ?? 4; - const checkpointTokens = - checkpoint.version === 3 - ? checkpoint.estimatedTokens - : estimateRuntimeEventsTokens( - [historyCompactCheckpointToRuntimeEvent(checkpoint)], - charsPerTokenResolved, - ); - const replayTokens = - checkpointTokens + estimateRuntimeEventsTokens(replayTail, charsPerTokenResolved); - const maxHistoryTokens = finitePositive( - options.maxHistoryEstimatedTokens ?? maxHistoryEstimatedTokens, - ); - if (maxHistoryTokens !== undefined && replayTokens > maxHistoryTokens) { - return { fits: false, checkpointTokens, replayTokens, reason: 'prefix_over_budget' }; - } - if (options.sourceReplayEvents) { - const sourceReplayTokens = estimateRuntimeEventsTokens( - options.sourceReplayEvents, - charsPerTokenResolved, - ); - if (replayTokens >= sourceReplayTokens) { - return { fits: false, checkpointTokens, replayTokens, reason: 'replacement_not_smaller' }; - } - } - return { fits: true, checkpointTokens, replayTokens }; -} - /** Replay the latest durable checkpoint when it exactly covers the ledger prefix. */ export function applyRuntimeEventHistoryCompact( events: readonly RuntimeEvent[], policy: HistoryCompactionPolicy | undefined, charsPerToken = 4, - maxHistoryEstimatedTokens?: number, - options: HistoryCompactionReplayOptions = {}, ): HistoryCompactionReplayResult { const checkpoint = policy?.enabled === true ? policy.checkpoint : undefined; if (!checkpoint) return { events: [...events], diagnosticPatch: {} }; @@ -495,32 +391,17 @@ export function applyRuntimeEventHistoryCompact( }), }; } - const headAnchor = - checkpoint.phase === 'mid_turn' - ? midTurnHeadAnchorEvent(checkpoint, match.coveredRuntimeEvents) - : undefined; - const replayTail = headAnchor - ? [headAnchor, ...match.successorRuntimeEvents] - : [...match.successorRuntimeEvents]; - const fit = evaluateHistoryCompactCheckpointReplay( - checkpoint, - replayTail, - charsPerToken, - maxHistoryEstimatedTokens, - { ...options, sourceReplayEvents: options.sourceReplayEvents ?? compactableEvents }, - ); - if (!fit.fits) { - return { - events: [...events], - diagnosticPatch: compactionDecisionDiagnosticPatch({ - stage: 'priorReplay', - sourceKind: 'runtimeEvents', - decision: 'failedOpen', - boundaryKind: 'historyCompact', - failOpenReason: fit.reason, - }), - }; - } + // A matching checkpoint always replays as `[block, tail]`: the fold chose + // the boundary structurally, and whether the result fits is the provider's + // answer, not a local estimate's (#4559). The token figures below are + // diagnostics only. + const checkpointTokens = + checkpoint.version === 3 + ? checkpoint.estimatedTokens + : estimateRuntimeEventsTokens( + [historyCompactCheckpointToRuntimeEvent(checkpoint)], + charsPerToken, + ); return { events: projectHistoryCompactCheckpointReplay( checkpoint, @@ -541,7 +422,7 @@ export function applyRuntimeEventHistoryCompact( bodySha256: [checkpoint.coverage.sourceDigest], }, estimatedTokensBefore: estimateRuntimeEventsTokens(match.coveredRuntimeEvents, charsPerToken), - estimatedTokensAfter: fit.checkpointTokens, + estimatedTokensAfter: checkpointTokens, }), }; } diff --git a/packages/runtime/src/memory-extraction.ts b/packages/runtime/src/memory-extraction.ts index 7a935319c5..194b3dfd57 100644 --- a/packages/runtime/src/memory-extraction.ts +++ b/packages/runtime/src/memory-extraction.ts @@ -1397,20 +1397,6 @@ function preparedMemoryRangeFits( ); } -function memoryRangeEventWeight({ event }: MemoryExtractionEventEntry): number { - if ( - !event.partial && - event.content?.kind === 'text' && - ((event.role === 'user' && event.author === 'user') || - (event.role === 'model' && event.author === 'agent')) - ) { - // Text appears once in the interpretation messages and, for user text, once - // more in the bounded evidence index when it cannot be referenced by position. - return Math.max(1, event.content.text.length * (event.role === 'user' ? 2 : 1)); - } - return 1; -} - function memoryRequestFits( snapshot: MemoryExtractionSourceSnapshot, prompt: string, @@ -1486,6 +1472,20 @@ function safeJsonLength(value: unknown): number { } } +function memoryRangeEventWeight({ event }: MemoryExtractionEventEntry): number { + if ( + !event.partial && + event.content?.kind === 'text' && + ((event.role === 'user' && event.author === 'user') || + (event.role === 'model' && event.author === 'agent')) + ) { + // Text appears once in the interpretation messages and, for user text, once + // more in the bounded evidence index when it cannot be referenced by position. + return Math.max(1, event.content.text.length * (event.role === 'user' ? 2 : 1)); + } + return 1; +} + function memoryEvidenceContainsSensitiveText( evidence: readonly { readonly events: readonly MemoryExtractionEventEntry['event'][] }[], ): boolean { diff --git a/packages/runtime/src/model-factory.ts b/packages/runtime/src/model-factory.ts index 43080a6d21..08ee365411 100644 --- a/packages/runtime/src/model-factory.ts +++ b/packages/runtime/src/model-factory.ts @@ -61,6 +61,7 @@ import { resolveModelRuntime, type ResolvedModelRuntime } from './model-runtime. import { runtimeProviderName, type RuntimeProviderAdapter } from './provider-runtime-policy.js'; import { openAiCodexHeaders } from './subscription-auth.js'; import { createRequestCustomizationFetch } from './request-customization-fetch.js'; +import { createStreamUsageFallbackFetch } from './stream-usage-fallback-fetch.js'; export interface ModelFactoryInput { connection: RuntimeExecutionConnection; @@ -236,8 +237,14 @@ export function getAIModel(input: ModelFactoryInput): LanguageModelV4 { name: runtimeProviderName(adapter, connection), apiKey, baseURL, - includeUsage: adapter.includeUsage, - fetch: reasoningTransport.fetch, + // Ask every Chat Completions server for stream usage unless the + // registry opts a provider out. Usage is the only signal the runtime's + // context handling reads (#4559): without `stream_options.include_usage` + // an OpenAI-compatible relay or a local Ollama returns none, and the + // proactive compaction baseline, the eviction check, and the usage + // indicator all go dark for exactly the connections that need them. + includeUsage: adapter.includeUsage ?? true, + fetch: createStreamUsageFallbackFetch(reasoningTransport.fetch, baseURL), transformRequestBody, ...(adapter.replayAssistantReasoningDetails ? { metadataExtractor: reasoningDetailsMetadataExtractor() } diff --git a/packages/runtime/src/openai-codex-history-compactor.ts b/packages/runtime/src/openai-codex-history-compactor.ts index 412e5e2f10..8cb442d4a8 100644 --- a/packages/runtime/src/openai-codex-history-compactor.ts +++ b/packages/runtime/src/openai-codex-history-compactor.ts @@ -30,7 +30,6 @@ import { } from './history-compact-checkpoint.js'; import type { RuntimeEvent } from '@maka/core/runtime-event'; import type { ModelMessage } from './model-protocol.js'; -import { fitHistoryCompactMessages } from './history-compact-input-fit.js'; import { admitProviderReasoningReplayItems, buildRuntimeEventModelReplayPlan, @@ -39,7 +38,7 @@ import { } from './model-history.js'; import { withProviderStreamTracking } from './provider-request-telemetry.js'; import { effectiveReplayToolResultOutput } from './durable-tool-result-projection.js'; -import { providerFailureDiagnostic } from './provider-error-classification.js'; +import { classifyError, providerFailureDiagnostic } from './provider-error-classification.js'; export interface BuildOpenAiCodexHistoryCompactorOptions { resolveModel: () => unknown; @@ -91,10 +90,9 @@ export function buildOpenAiCodexHistoryCompactor(options: BuildOpenAiCodexHistor if (canContinuePrevious) { projectedMessages.unshift(historyCompactCheckpointToModelMessage(previous)); } - const messages = fitHistoryCompactMessages(projectedMessages, { - maxInputEstimatedTokens: input.inputBudget?.maxEstimatedTokens, - charsPerToken: input.inputBudget?.charsPerToken, - }); + // Whether this input fits the compaction endpoint is its own answer; + // nothing is trimmed on a local estimate beforehand (#4559). + const messages = projectedMessages; const providerRequestTracker = input.providerRequestTracker; const ai = await loadAiSdkModule(); @@ -133,6 +131,9 @@ export function buildOpenAiCodexHistoryCompactor(options: BuildOpenAiCodexHistor return state; } catch (error) { if (error instanceof HistoryCompactSummarizerError) throw error; + if (classifyError(error) === 'ContextLength') { + throw new HistoryCompactSummarizerError('input_too_large', { cause: error }); + } throw new HistoryCompactSummarizerError('provider_error', { cause: error }); } }; diff --git a/packages/runtime/src/stream-usage-fallback-fetch.ts b/packages/runtime/src/stream-usage-fallback-fetch.ts new file mode 100644 index 0000000000..059bd441da --- /dev/null +++ b/packages/runtime/src/stream-usage-fallback-fetch.ts @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +/** + * One-time `stream_options` retreat for strict OpenAI-compatible servers. + * + * Maka asks every Chat Completions server for stream usage because usage is + * the only signal its context handling reads (#4559). A relay or gateway that + * rejects unknown request fields (older vLLM builds, some proxies) answers 400 + * to every streaming request instead, and a user cannot switch the field off + * without a code change. So the first such rejection is answered once, in the + * open: resend the same request without the field and remember that this + * endpoint cannot report usage, rather than failing every request or silently + * never asking. The connection then runs without a baseline — the composer + * indicator shows no usage — which is the honest state for a server that + * cannot report it. + * + * The retreat is deliberately narrow. Only a 400 whose body names the field + * counts; any other rejection is the provider's answer and is returned + * untouched. + */ + +type FetchLike = typeof globalThis.fetch; + +/** Endpoints observed to reject the field. Process-lifetime, per base URL. */ +const endpointsWithoutStreamUsage = new Set(); + +const FIELD_PATTERN = /stream_options|include_usage/i; + +function withoutStreamOptions(body: string): string | undefined { + let parsed: unknown; + try { + parsed = JSON.parse(body); + } catch { + return undefined; + } + if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) return undefined; + const record = parsed as Record; + if (!('stream_options' in record)) return undefined; + const { stream_options: _dropped, ...rest } = record; + return JSON.stringify(rest); +} + +function requestBodyText(init: RequestInit | undefined): string | undefined { + const body = init?.body; + return typeof body === 'string' ? body : undefined; +} + +/** Test seam: forget every remembered endpoint. */ +export function resetStreamUsageFallbackMemory(): void { + endpointsWithoutStreamUsage.clear(); +} + +export function createStreamUsageFallbackFetch(baseFetch: FetchLike, baseUrl: string): FetchLike { + return async (input, init) => { + const remembered = endpointsWithoutStreamUsage.has(baseUrl); + const body = requestBodyText(init as RequestInit | undefined); + if (remembered && body !== undefined) { + const stripped = withoutStreamOptions(body); + if (stripped !== undefined) { + return baseFetch(input, { ...(init as RequestInit), body: stripped }); + } + } + const response = await baseFetch(input, init); + if (remembered || response.status !== 400 || body === undefined) return response; + const stripped = withoutStreamOptions(body); + if (stripped === undefined) return response; + let text = ''; + try { + text = await response.clone().text(); + } catch { + return response; + } + if (!FIELD_PATTERN.test(text)) return response; + endpointsWithoutStreamUsage.add(baseUrl); + return baseFetch(input, { ...(init as RequestInit), body: stripped }); + }; +} diff --git a/packages/ui/src/conversation-copy.ts b/packages/ui/src/conversation-copy.ts index 3042b31f3c..81c8274df1 100644 --- a/packages/ui/src/conversation-copy.ts +++ b/packages/ui/src/conversation-copy.ts @@ -320,6 +320,10 @@ export interface ConversationCopy { systemNotes: { contextCompacted: string; contextCompactionFailedOpen: string; + contextProviderDropping: string; + contextWindowSuggestion: (tokens: number, declared: number | undefined) => string; + contextWindowOverrun: (used: number, declared: number) => string; + contextReportedWindowExceeded: (used: number, reported: number) => string; stepLimit: string; }; }; @@ -533,6 +537,15 @@ const CONVERSATION_COPY = { systemNotes: { contextCompacted: '已压缩较早的对话内容，以适应模型上下文窗口。', contextCompactionFailedOpen: '上下文摘要失败；本轮已在未生成新摘要的情况下继续。', + contextProviderDropping: '供应商在丢弃或改写上下文（追加了内容但用量未增长）。在连接设置里为该模型声明上下文窗口，让 Maka 先行压缩。', + contextWindowSuggestion: (tokens, declared) => + declared === undefined + ? `供应商拒绝了这次请求。该模型未声明上下文窗口；上次成功的用量约 ${tokens} tokens，可将窗口设为该值让 Maka 先行压缩。` + : `供应商拒绝了这次请求，但用量（约 ${tokens} tokens）低于你声明的窗口（${declared}）。声明值可能大于供应商实际窗口，建议下调到 ${tokens}。`, + contextWindowOverrun: (used, declared) => + `本次交换用了约 ${used} tokens，超过你声明的窗口（${declared}）：回复需要的空间比剩余的多。Maka 会在下一次请求前压缩；若希望回复保持完整，可调大窗口。`, + contextReportedWindowExceeded: (used, reported) => + `本次交换用了约 ${used} tokens，已超过该模型上报的窗口（${reported}），但供应商没有拒绝。你未声明窗口，Maka 因此不会主动压缩。在连接设置里声明一个窗口即可让它先行压缩。`, stepLimit: '已达到本轮工具步骤上限，任务可能尚未完成。发送“继续”即可接着处理。', }, }, @@ -691,6 +704,15 @@ const CONVERSATION_COPY = { systemNotes: { contextCompacted: 'Context compacted to keep this session within the model window.', contextCompactionFailedOpen: 'Context summary failed; the session continued without a new summary.', + contextProviderDropping: 'The provider is dropping or rewriting context (content was appended but usage did not grow). Declare a context window for this model in the connection settings so Maka compacts first.', + contextWindowSuggestion: (tokens, declared) => + declared === undefined + ? `The provider rejected this request. No context window is declared for this model; the last accepted usage was about ${tokens} tokens — set the window to that value so Maka compacts first.` + : `The provider rejected this request at about ${tokens} tokens, below your declared window (${declared}). The declared value is likely larger than the provider's; consider lowering it to ${tokens}.`, + contextWindowOverrun: (used, declared) => + `This exchange used about ${used} tokens against your declared window (${declared}): the reply needed more room than was left. Maka compacts before the next request; raise the window if the replies should stay whole.`, + contextReportedWindowExceeded: (used, reported) => + `This exchange used about ${used} tokens, past the ${reported} this model reports, and the provider accepted it without complaint. Nothing is declared, so Maka does not compact on its own. Declare a context window in the connection settings to have it compact first.`, stepLimit: 'Reached the configured step limit. The task may be incomplete. Send “continue” to resume.', }, }, diff --git a/packages/ui/src/materialize.ts b/packages/ui/src/materialize.ts index 02dc73fde9..ff14b59b2c 100644 --- a/packages/ui/src/materialize.ts +++ b/packages/ui/src/materialize.ts @@ -140,10 +140,43 @@ export interface ToolActivityItem { shellRunSource?: "owned" | "unavailable"; } -function systemNoteLabel(kind: string, locale: UiLocale): string { +function systemNoteLabel(kind: string, data: unknown, locale: UiLocale): string { const copy = getConversationCopy(locale).messages.systemNotes; if (kind === "context_compacted") return copy.contextCompacted; if (kind === "context_compaction_failed_open") return copy.contextCompactionFailedOpen; + if (kind === "context_provider_dropping") return copy.contextProviderDropping; + if (kind === "context_reported_window_exceeded") { + const exceeded = data as + | { usedTokens?: unknown; reportedContextWindow?: unknown } + | undefined; + const used = typeof exceeded?.usedTokens === "number" ? exceeded.usedTokens : 0; + const reported = + typeof exceeded?.reportedContextWindow === "number" ? exceeded.reportedContextWindow : 0; + return copy.contextReportedWindowExceeded(used, reported); + } + if (kind === "context_window_overrun") { + const overrun = data as + | { usedTokens?: unknown; declaredContextWindow?: unknown } + | undefined; + const used = typeof overrun?.usedTokens === "number" ? overrun.usedTokens : 0; + const declared = + typeof overrun?.declaredContextWindow === "number" ? overrun.declaredContextWindow : 0; + return copy.contextWindowOverrun(used, declared); + } + if (kind === "context_window_suggestion") { + const suggestion = data as + | { suggestedContextWindow?: unknown; declaredContextWindow?: unknown } + | undefined; + const tokens = + typeof suggestion?.suggestedContextWindow === "number" + ? suggestion.suggestedContextWindow + : 0; + const declared = + typeof suggestion?.declaredContextWindow === "number" + ? suggestion.declaredContextWindow + : undefined; + return copy.contextWindowSuggestion(tokens, declared); + } if (kind === "step_limit") return copy.stepLimit; return kind; } @@ -187,7 +220,7 @@ export function materializeChat( items.push({ id: message.id, role: "system", - text: systemNoteLabel(message.kind, locale), + text: systemNoteLabel(message.kind, message.data, locale), ts: message.ts, }); } @@ -772,7 +805,7 @@ export function materializeTurns( turn.notes.push({ id: message.id, role: "system", - text: systemNoteLabel(message.kind, locale), + text: systemNoteLabel(message.kind, message.data, locale), ts: message.ts, }); } else if (message.type === "token_usage") {