fix(runtime): stop estimating context fit; the provider decides, the Maka window is a user target - #4574
fix(runtime): stop estimating context fit; the provider decides, the Maka window is a user target#4574Joob1n wants to merge 13 commits into
Conversation
Astro-Han
left a comment
There was a problem hiding this comment.
The direction is the one from #4458 and I still hold it: fit is the provider's answer, the trigger is a real number, and the only irreversible outcome is the provider's rejection. The reduction is real, every symbol in "What is gone" is gone from docs and desktop too, and all six workspace suites, typecheck, lint and format pass locally on 372c1de9, and the three commits since (148326eb) touch none of the files cited below. What stops me approving is not the code, it is three places where the design as implemented does not do what the design says. Line-level fixes are inline.
The proactive trigger cannot fire against a correctly declared window. The trigger is baseline > declaredWindow, and the baseline is a request the provider accepted. If the user declares the provider's real window, no accepted request can exceed it, the fold never runs, and every session goes through rejection first. It fires only when the user declares something smaller, which is the retired reserve reintroduced as a number the user has to guess without being told. This is open point 2 of #4559, and I would settle it the other way: resolve the window as declared > relay profile > models.dev > /models, none only when all are absent; trigger on baseline > window − reserve; make the reserve a real number, the model's output limit plus the tool-result archive cap times pending tool calls, with a fraction of the window as the floor. Those are provider facts and Maka's own content policy, so the "real numbers only" principle holds. A default is not Maka deciding for the user, it is a default the user can override, and the alternative is that out of the box every fill costs a rejection and possibly a cut reply, which CLI users get without even the hint, because pi-transcript.ts drops the two new note kinds. The per-step check already gives mid-turn triggering; only the window and the threshold change.
The summarizer's provider does not get to decide. The design names four deciders, and one of them, "the summarizer's provider decides whether compaction input fits, input_too_large retreats the fold by half", has no producer. Only fitHistoryCompactMessages throws input_too_large, this PR removes that call from history-compact-summarizer.ts and openai-codex-history-compactor.ts, and both catches wrap every provider error as provider_error. So a session whose fold prefix is too large for the summarizer fails open on the first rejection, /compact included, and has no in-app way back under the window. That is worse than main, which at least estimated. The fix is small, classify ContextLength in both catches, but the regression must use a fake provider that rejects above N tokens; all five current input_too_large tests throw the error from a fixture, which is why this was invisible.
The compatibility contract. Two new system_note kinds and the new lastRequestAnchor shape land in packages/core closed allowlists, the same class of change #4486 bumped the epoch for, and the epoch is unchanged. A merge-base client handshakes fine (host-kernel.ts admits on strict equality) and fails decodeStoredMessage on the first transcript carrying a new note. main is on 96 now: rebase, take 97, and put back the three CHANGELOG sentences from #4486 that are still true (see inline).
One consequence of the first point that lives outside the diff: settings-provider-copy.ts still says "when empty, built-in metadata decides", and /context and the inspector still divide by resolveSelectedModelContextWindow, so an undeclared user sees 70% of 200k and never gets a fold. Whichever way the window question settles, the copy and the denominator have to follow the same rule.
On the test claim: five re-pointed tests pass unchanged on main's built sources (the no-size-gate replay, the two size-floor cases, the no-usage case, fails open after repeated input-too-large retreat), and two rules lost their only coverage with the deletions while still existing, the memory gate's unavailable outcome and the rule that a synthetic /compact usage row must not shadow the real anchor. The remaining persisted-anchor test uses an anchor below the window, so it passes regardless of the run-header check.
Evidence boundary: static read against main 61224f51; all suites run locally; new tests also run against main's built sources in a scratch copy; no live provider.
AI-assisted review: drafted with Maka; I verified the trigger arithmetic, the input_too_large producers, the epoch state and the size-floor inputs myself.
简体中文
方向就是 #4458 上讨论定下的,我仍然支持:放不放得下由 provider 回答,压缩触发用真实数字,唯一不可逆的结果是 provider 的拒绝。删得也干净,「What is gone」里的每个符号在 docs 和 desktop 里都没有残留,本地六个 workspace 的测试、typecheck、lint、format 全绿;之后的三个提交(148326eb)没有碰到下面引用的任何文件。让我暂时不 approve 的不是代码本身,而是三处实现和设计说的不是一回事。能局部修的都放在行内评论里。
主动触发对着正确声明的窗口永远打不响。 触发条件是「上次请求的 input+output 大于声明窗口」,而上次请求是 provider 已经接受的。如果用户把窗口声明成 provider 的真实大小,任何被接受的请求都不可能超过它,主动压缩一次也不会触发,每个会话都要先撞一次拒绝。只有用户把窗口声明得比真实值小它才触发,这等于把删掉的 reserve 变成了一个要用户自己猜、又不告诉他的数字。这是 #4559 的 open point 2,我的意见相反:窗口按「用户声明 > relay profile > models.dev 元数据 > /models 报告」解析,全都没有才算无;触发条件改成「上次用量 > 窗口 − reserve」;reserve 用真实数字:模型的 output limit(下一步回复最多这么大)加上工具结果归档上限乘以待处理的工具调用数,再用窗口的固定比例兜底。这些都是 provider 的事实和 Maka 自己的内容策略,「只用真实数字」的原则不受影响。给一个默认值不是替用户决定,用户随时可以覆盖;不给的后果是开箱用户每次填满都要付一次拒绝、可能还拿到被截断的回复,CLI 用户连提示都看不到,因为 pi-transcript.ts 把两个新 note kind 丢掉了。turn 内每一步都检查的逻辑已经有了,只需要改窗口和阈值。
summarizer 的 provider 并没有拿到决定权。 设计列了四个决定者,其中「summarizer 的 provider 决定压缩输入放不放得下,input_too_large 就把范围退半」这一条没有产生者。全仓只有 fitHistoryCompactMessages 会抛 input_too_large,本 PR 把它从 history-compact-summarizer.ts 和 openai-codex-history-compactor.ts 里删掉了,两处的 catch 又把所有 provider 错误一律包成 provider_error。于是一个长会话的折叠前缀如果对 summarizer 也太大,第一次拒绝就整体放弃,/compact 走同一条路也一样,会话在 app 内没有回到窗口内的办法,比 main 还差,main 至少还估算一下。修法很小,两个 catch 里判断 ContextLength 再抛,但回归测试必须用「输入超过 N tokens 就拒绝」的假 provider;现在五个相关测试全是 fixture 手抛这个错误,所以这个洞看不见。
兼容契约。 两个新的 system_note kind 和新的 lastRequestAnchor 形状都进了 packages/core 的闭合 allowlist,和 #4486 推 epoch 的那次是同一类改动,但 epoch 没动。merge-base 上的旧 client 握手能通过(host-kernel.ts 按严格相等准入),然后在第一条带新 note 的 transcript 上解码失败。main 现在已经是 96:rebase 取 97,并把 #4486 写的、仍然成立的三句 CHANGELOG 加回来(见行内)。
第一点在 diff 之外的一个后果:settings-provider-copy.ts 的帮助文案「留空跟随内置元数据」现在不成立,/context 和 inspector 的分母也还是 resolveSelectedModelContextWindow,未声明的用户会看到「已用 70% / 200k」却永远等不到压缩。窗口问题无论怎么定,文案和分母都要跟同一条规则。
关于测试的说法:五条 re-pointed 测试在 main 的构建产物上原样通过(无 size gate 的 replay、两条体量下限、无 usage、退半后 fail open);两条规则随删测试失去了唯一覆盖但仍然存在:memory gate 返回 unavailable 的处理,以及手动 /compact 写的合成 usage 行不得遮蔽真实 anchor。剩下那条持久 anchor 测试的 anchor 低于窗口,不管 run header 合不合法都过。
| // window, or the provider cut the previous reply at its output limit. | ||
| const lengthFold = state.pendingLengthFold; | ||
| state.pendingLengthFold = false; | ||
| const overWindow = |
There was a problem hiding this comment.
baselineTokens is a request the provider accepted, so with the window declared at the provider's real size this is never true; the fold only runs when the user declares something smaller. That is the reserve in disguise. baseline > window − reserve, with the reserve from output limit plus the tool-result cap, is what makes this fire before the rejection. First point in the main comment.
| @@ -335,7 +281,12 @@ export async function planHistoryCompaction( | |||
| } catch (error) { | |||
| if (error instanceof HistoryCompactSummarizerError) { | |||
| if (error.reason === 'input_too_large') { | |||
There was a problem hiding this comment.
Dead branch: nothing throws input_too_large any more. fitHistoryCompactMessages was its only producer and both summarizer catches now wrap provider errors as provider_error. Classify ContextLength with classifyError in history-compact-summarizer.ts and openai-codex-history-compactor.ts so this retreat has a producer, and test it with a fake provider that rejects above N tokens.
| stepUsage !== undefined && | ||
| Number.isFinite(stepUsage.inputTokens) && | ||
| stepUsage.inputTokens > 0 && | ||
| stepUsage.inputTokens < midTurnState.baselineTokens |
There was a problem hiding this comment.
Baseline includes last step's reasoning; on wires that do not resend it (gpt-5.x), next input ≈ previous input + tool call + result, which is below input + output whenever reasoning exceeded the appended content, the normal case. Compare against last step's input alone; keep input + output for the trigger.
| id: this.newId(), | ||
| turnId, | ||
| ts: this.now(), | ||
| kind: 'context_window_suggestion', |
There was a problem hiding this comment.
Written before recoverFromOverflowError runs, so a send that overflows once, folds and completes still tells the user the provider refused. #4559 places this under "already folded". Move it after the recovery verdict; overflow-reactive-recovery.test.ts:1834 (tool, overflow, done) currently pins the wrong order. And suggestedContextWindow: baselineTokens sits on the strict > boundary, so a user who applies it still needs one more rejection.
| ) { | ||
| return 'malformed_summary_too_small_for_fold'; | ||
| } | ||
| const usage = foldContext?.summarizerUsage; |
There was a problem hiding this comment.
On a roll-forward the summarizer is sent previous summary + new events, so inputTokens is the increment while the checkpoint replaces the whole covered span; in steady state this floor never trips and a fifty-token summary after a small increment is persisted. The new test passes on main because the stub's usage is unrelated to the messages. Feed it a span-sized real number (previous checkpoint's compactCallInputTokens plus this increment) or drop the floor openly.
| @@ -225,7 +225,6 @@ export function buildHistoryCompactCheckpoint( | |||
| if (!providerState && input.summaryFormat !== 'legacy_freeform') { | |||
| const defect = findCheckpointSummaryDefect(summary!, { | |||
| coveredRuntimeEvents: input.coveredRuntimeEvents, | |||
There was a problem hiding this comment.
The comment above says this re-runs the size floor, but no summarizerUsage is passed so the floor is skipped, and coveredRuntimeEvents is now required and unread by the validator. Same for the external-producer branch in history-compaction.ts. Either pass a real number or delete the field, the three constructions and the comment.
| and SessionEvent-to-RuntimeEvent conversion remains a pure mapper. | ||
| - Retired the Task Ledger domain: SessionTodo is now the sole authority for in-session work items, and the operational-state schema drops the `workflow_task_ledger_events` table on first open. **Unfinished Tasks are not migrated and are permanently deleted.** This affects workspaces last opened by `v0.1.0` through `v0.1.11`, `cli-v0.1.0-beta.1`, `v0.2.0-incubating-rc1`, or a `v0.2.0-dev` build; those releases wrote Tasks to a table that no shipped build ever bridged into SessionTodo. Before opening such a workspace with this build, finish or export the Tasks you still need, or copy the workspace's `runtime.sqlite` aside — the migration removes the only live copy, so afterwards recovery requires a backup made in advance. | ||
| - Let the provider decide whether a request fits, and anchored the estimate that decides when to compact on the last request the provider actually counted. `token_usage` records now persist that anchor under a new `lastRequestAnchor` key. **Sessions this build writes do not open in earlier releases:** those decode `token_usage` against a closed allowlist, so the unknown key fails the record and, with it, the Session that contains it. Downgrading therefore needs a copy of the workspace's `runtime.sqlite` taken before the upgrade. Retired with the local verdict: nothing produces the `context_budget_exhausted` stop reason any more — a request that really is too large is compacted and retried once, then reported as a `context_overflow` provider error — though sessions that already recorded it still decode and present. The Runtime Host compatibility epoch moves to 94. | ||
| - Let the provider decide whether a request fits. Proactive compaction now uses only a user-declared Maka window and the previous accepted request's provider-reported `inputTokens + outputTokens`; no declaration means no proactive capacity threshold. `/models` and generated model metadata are display hints, not limits. `token_usage` records persist the last-request anchor under `lastRequestAnchor`; its new `{ inputTokens, outputTokens }` shape still decodes the retired `payloadChars` key from older sessions. Requests that are too large are compacted and retried once after a real provider rejection, then reported as a `context_overflow` provider error. New provider-dropping and context-window suggestion system notes explain provider-side context changes. |
There was a problem hiding this comment.
Three sentences from #4486 were dropped and are still true: sessions from this build do not open in earlier releases and the runtime.sqlite copy must be taken before upgrading; context_budget_exhausted is no longer produced but still decodes; the epoch line (97 after rebase). The body says the allowlist cost is "already noted in CHANGELOG.md", and it no longer is.
148326e to
a650762
Compare
Review findings from apache#4574, each with a test that fails on the previous commit: - The proactive trigger now reserves the model's declared output limit (`connection.models[].maxOutputTokens` or generated metadata, a provider fact; 0 when none is declared) and fires on `>=`: with the window declared at the provider's real size, an accepted request alone can never exceed it, and the reply the next request must leave room for is what tips it. The window itself stays the user's declaration. - The summarizer's provider is a decider again: a context-length rejection from the summary call (or the Codex compaction call) surfaces as `input_too_large`, the reason the planner retreats on. Removing the estimated pre-fit had left that path without a producer. - The eviction check compares input against input: the previous reply's reasoning is not resent on every wire, so input + output is not a floor. - The window suggestion is written after recovery, only when the turn surfaces the rejection, and names the last accepted total, which a fold does not clear. It is no longer a number on the `>` boundary. - The size floor judges an initial fold only; on a roll-forward the summarizer's input is the increment, not the covered span. - `CheckpointSummaryFoldContext` carries usage only; the builder and copy seams validate structure and truncation, which is all they can. - The CLI transcript renders the two new note kinds. - Restored coverage: a synthetic /compact usage row does not shadow the real anchor; the run-header check is exercised with an anchor above the window. - CHANGELOG carries the downgrade and epoch sentences again. The Host's reported `contextWindow` stays the model's window: it is a protocol fact clients already display, and the declared window has its own surface in the composer indicator. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
|
Thanks for the read — every finding was real. Fixes are in Trigger arithmetic. You are right that Window source — kept as declared-only, deliberately. This is the one point where I did not take the suggestion, and it is the issue author's call rather than mine: the Maka window is a target the user chooses to stay under, not a fact about the model, so the model's reported window is shown beside the setting with one click to apply (#4575) but never becomes the threshold on its own. The cost you name is real and stated in the description: with nothing declared, the first fill costs one rejection and possibly one cut reply. The
Epoch and CHANGELOG. Bumped to 97 on rebase and the three sentences are back. Inline items: eviction check compares input with input; the size floor applies to the initial fold only (the roll-forward input is the increment, as you say) and Two things I left as they were, with reasons. The Host's |
a650762 to
a67327b
Compare
Review findings from apache#4574, each with a test that fails on the previous commit: - The proactive trigger now reserves the model's declared output limit (`connection.models[].maxOutputTokens` or generated metadata, a provider fact; 0 when none is declared) and fires on `>=`: with the window declared at the provider's real size, an accepted request alone can never exceed it, and the reply the next request must leave room for is what tips it. The window itself stays the user's declaration. - The summarizer's provider is a decider again: a context-length rejection from the summary call (or the Codex compaction call) surfaces as `input_too_large`, the reason the planner retreats on. Removing the estimated pre-fit had left that path without a producer. - The eviction check compares input against input: the previous reply's reasoning is not resent on every wire, so input + output is not a floor. - The window suggestion is written after recovery, only when the turn surfaces the rejection, and names the last accepted total, which a fold does not clear. It is no longer a number on the `>` boundary. - The size floor judges an initial fold only; on a roll-forward the summarizer's input is the increment, not the covered span. - `CheckpointSummaryFoldContext` carries usage only; the builder and copy seams validate structure and truncation, which is all they can. - The CLI transcript renders the two new note kinds. - Restored coverage: a synthetic /compact usage row does not shadow the real anchor; the run-header check is exercised with an anchor above the window. - CHANGELOG carries the downgrade and epoch sentences again. The Host's reported `contextWindow` stays the model's window: it is a protocol fact clients already display, and the declared window has its own surface in the composer indicator. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
|
Rebased on |
There was a problem hiding this comment.
方向我赞成:把“请求是否 fit”的裁决权完全交给 provider,把本地侧收缩为“真实 usage + 用户目标只触发可逆 fold”,比多把估算尺同时参与决策更干净。PR 的主要因果链也基本成立。不过在 approve 前,我认为还有下面几个点需要收口:
-
[P1] compatibility epoch 与最新 main 撞号。 当前
main@898b86d已经把RUNTIME_HOST_COMPATIBILITY_EPOCH升到 99,用于 ScheduledTask 的 Connection identity;本 PR 也独立使用 99 表示两个新 system note 和lastRequestAnchorshape。两边是不同的不兼容协议变化。rebase 后不能只解决文本冲突并继续保留 99,需要把合并结果升到 100,同步注释、CHANGELOG 和 epoch test;否则能力不同的 client/host 可能错误通过严格相等握手。 -
[P2] repair 请求里的 ContextLength 会被吞成初次格式错误。
history-compact-summarizer.ts:206-243中,初次 summary malformed 后,repair 使用更长的 system prompt。如果初次调用刚好 fit、repair 因新增 instruction 被 provider 拒绝,内层 catch 会把该错误包装成初次的malformed_summary_*;外层看到它已经是HistoryCompactSummarizerError后直接重抛,不会再分类为input_too_large。planner 因此不会执行 halve-and-retry,和“summarizer provider decides”不一致。建议在 repair catch 中保留 ContextLength 分类,并补一个“initial malformed, repair overflow”的回归测试。 -
[P2] 架构文档仍描述修复前的 trigger。 中英文文档目前都写成“上次
inputTokens + outputTokens超过用户窗口时触发”;实现已经是baseline + model.maxOutputTokens >= declaredWindow(ai-sdk-compaction.ts:896-908)。这是核心规则,应同步两份文档,避免后续实现和测试重新漂移。 -
Issue closure 建议调整。 #4559 的 proposal 还包含 missing-usage hint、unclassified overflow 的 compact-and-retry action、metadata window 一键采用和 composer indicator。本 PR 没有 missing-usage note,unclassified action 明确留在系列之外,UI 又拆到 #4575/#4576。拆 PR 没问题,但此时
Closes #4559会让完整验收范围过早关闭;建议改成Refs #4559,或先明确把 #4559 收缩为 runtime-only。
从 Occam 的角度,我的判断是:架构方向已收敛,问题定义尚未完全收敛。 最小因果主线已经很清楚:
history grows
→ user target may request a reversible fold
→ provider outcome decides actual fit
→ canonical history is never rejected or mutated by an estimate
但“provider decides”尚未覆盖 provider 沉默的情况:PR 自己的 Ollama 实测已经显示跨 turn usage plateau / silent truncation,而当前 dropping note 只检查同 turn 的 step ≥ 1。建议把无 usage、无法分类以及 silent eviction 明确定义成可见的 degraded mode;否则“未声明 window 时系统提供什么保证”仍是开放语义。
另外,includeUsage ?? true 现在会给所有 OpenAI-compatible Chat server 发送 stream_options.include_usage。现有 conformance test 只覆盖支持该字段的 fake server;最好再补一个严格 relay 拒绝该字段时的契约测试,明确 fail、fallback 或 opt-out 行为。
CI 当前是绿的,diff check 也干净;上面第 1 项是基于最新 main 的当前 merge blocker,第 2 项是静态可达的 provider-boundary bug。修完这些,我认为核心 runtime 改动可以继续推进。
a67327b to
6c5e53d
Compare
Review findings from apache#4574, each with a test that fails on the previous commit: - The proactive trigger now reserves the model's declared output limit (`connection.models[].maxOutputTokens` or generated metadata, a provider fact; 0 when none is declared) and fires on `>=`: with the window declared at the provider's real size, an accepted request alone can never exceed it, and the reply the next request must leave room for is what tips it. The window itself stays the user's declaration. - The summarizer's provider is a decider again: a context-length rejection from the summary call (or the Codex compaction call) surfaces as `input_too_large`, the reason the planner retreats on. Removing the estimated pre-fit had left that path without a producer. - The eviction check compares input against input: the previous reply's reasoning is not resent on every wire, so input + output is not a floor. - The window suggestion is written after recovery, only when the turn surfaces the rejection, and names the last accepted total, which a fold does not clear. It is no longer a number on the `>` boundary. - The size floor judges an initial fold only; on a roll-forward the summarizer's input is the increment, not the covered span. - `CheckpointSummaryFoldContext` carries usage only; the builder and copy seams validate structure and truncation, which is all they can. - The CLI transcript renders the two new note kinds. - Restored coverage: a synthetic /compact usage row does not shadow the real anchor; the run-header check is exercised with an anchor above the window. - CHANGELOG carries the downgrade and epoch sentences again. The Host's reported `contextWindow` stays the model's window: it is a protocol fact clients already display, and the declared window has its own surface in the composer indicator. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
|
Thanks — the first three are real and are fixed in 1. Epoch. 2. Repair-path ContextLength. Confirmed: the repair catch wrapped every failure under the initial defect, so a context-length rejection of the longer repair prompt never reached the planner as 3. Docs. Both 4. Issue closure. Changed to On the two broader points:
|
6ce7d73 to
e10dd49
Compare
Review findings from apache#4574, each with a test that fails on the previous commit: - The proactive trigger now reserves the model's declared output limit (`connection.models[].maxOutputTokens` or generated metadata, a provider fact; 0 when none is declared) and fires on `>=`: with the window declared at the provider's real size, an accepted request alone can never exceed it, and the reply the next request must leave room for is what tips it. The window itself stays the user's declaration. - The summarizer's provider is a decider again: a context-length rejection from the summary call (or the Codex compaction call) surfaces as `input_too_large`, the reason the planner retreats on. Removing the estimated pre-fit had left that path without a producer. - The eviction check compares input against input: the previous reply's reasoning is not resent on every wire, so input + output is not a floor. - The window suggestion is written after recovery, only when the turn surfaces the rejection, and names the last accepted total, which a fold does not clear. It is no longer a number on the `>` boundary. - The size floor judges an initial fold only; on a roll-forward the summarizer's input is the increment, not the covered span. - `CheckpointSummaryFoldContext` carries usage only; the builder and copy seams validate structure and truncation, which is all they can. - The CLI transcript renders the two new note kinds. - Restored coverage: a synthetic /compact usage row does not shadow the real anchor; the run-header check is exercised with an anchor above the window. - CHANGELOG carries the downgrade and epoch sentences again. The Host's reported `contextWindow` stays the model's window: it is a protocol fact clients already display, and the declared window has its own surface in the composer indicator. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
|
Rebased on |
likun666661
left a comment
There was a problem hiding this comment.
复查了最新 head e10dd49a4。上一轮的主要项都已正确落地:epoch 102 对应 merge base 101;repair-path ContextLength 保留为 input_too_large 且有回归;两份文档的 trigger 公式已更新;PR body 改成了 Refs #4559;strict relay 的一次 400 / 不静默重试契约也已固定。CI 当前是绿的,PR 可干净合并。
还有两个行为/scope 问题和一个文档漂移需要收口:
-
[P1]
#4559仍会被 closing keyword 关闭。 PR body 虽然已经改成Refs #4559,但首个 commit20f01b862的 message body 仍然包含Closes #4559。本仓库只允许 squash merge,并配置为squash_merge_commit_message: COMMIT_MESSAGES;因此这个 closing keyword 会进入 default-branch 的 squash commit,仍会关闭 issue。既然约定是整个 #4574/#4575/#4576 系列落地并完成剩余 triage 后再关闭,请把该 commit message amend 为Refs #4559(或确保最终 squash message 中完全没有 closing keyword),而不只是修改 PR body。 -
[P2] provider-dropping 实现检测的是“严格下降”,对外契约写的是“没有增长”。
packages/runtime/src/ai-sdk-backend.ts:2170-2182当前条件是:stepUsage.inputTokens < lastStepInputTokens
但 PR summary、commit message、CLI/UI copy 和新开的 #4623 都把信号定义为 usage “did not grow / stops growing or drops”。本 PR 自己的 Ollama 证据正是输入在 3,716 持平 plateau;严格
<不会对这个案例写 note。当前测试也只有 100 → 50 的 decrease,没有 equal-usage case。如果 observable contract 确实是“append-only request 的 input 没增长”,这里应使用
<=并补一个 equal-usage regression。#4623 的 PR D 也不应只定义成 “below persisted anchor”,否则同一个跨-turn plateau 仍不会被检测。若等值被刻意排除,则应反过来收窄 PR、copy 和 #4623 的声明,并说明为什么 plateau 不是可靠信号;现在代码和承诺二者只能有一个成立。 -
[P3] 中文架构文档仍有两处旧的 size-policy 语义。
docs/architecture/llm-compaction-events-log-projection-draft.zh-CN.md:88-93的公式仍写current context budget,而英文已改成user-declared capacity and provider outcome;同文档:121-130仍把 checkpoint 的必要问题写成“它在当前 token policy 下仍然能否进入 prompt?”,英文已改为 source/provider-state identity。后文已经明确删除 replay size verdict,因此这两处应与英文及最终实现同步。
除了以上三项,上一轮要求的修复我没有再发现回归。前两项处理后,我认为这轮可以进入 approve。
e10dd49 to
2cbc332
Compare
Review findings from apache#4574, each with a test that fails on the previous commit: - The proactive trigger now reserves the model's declared output limit (`connection.models[].maxOutputTokens` or generated metadata, a provider fact; 0 when none is declared) and fires on `>=`: with the window declared at the provider's real size, an accepted request alone can never exceed it, and the reply the next request must leave room for is what tips it. The window itself stays the user's declaration. - The summarizer's provider is a decider again: a context-length rejection from the summary call (or the Codex compaction call) surfaces as `input_too_large`, the reason the planner retreats on. Removing the estimated pre-fit had left that path without a producer. - The eviction check compares input against input: the previous reply's reasoning is not resent on every wire, so input + output is not a floor. - The window suggestion is written after recovery, only when the turn surfaces the rejection, and names the last accepted total, which a fold does not clear. It is no longer a number on the `>` boundary. - The size floor judges an initial fold only; on a roll-forward the summarizer's input is the increment, not the covered span. - `CheckpointSummaryFoldContext` carries usage only; the builder and copy seams validate structure and truncation, which is all they can. - The CLI transcript renders the two new note kinds. - Restored coverage: a synthetic /compact usage row does not shadow the real anchor; the run-header check is exercised with an anchor above the window. - CHANGELOG carries the downgrade and epoch sentences again. The Host's reported `contextWindow` stays the model's window: it is a protocol fact clients already display, and the declared window has its own surface in the composer indicator. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
|
All three are fixed on 1. Closing keyword in the commit body. You are right that the PR body is not where this is decided under squash merge with 2. #4623 is updated to match: PR D's rule is "at or below the persisted anchor", not "below", for the same reason. 3. Chinese document drift. Both passages are synced with the English text: the materialization inputs now read 用户声明的 capacity 与 provider 结果, and the checkpoint's replay question is source/provider-state identity rather than the retired token policy. One note on scope: both documents still ask "which high-water decision produced it" in that same list. It is stale in both languages equally, so I left it rather than widening this PR's document diff; say the word and I will fold it in here, otherwise it goes with the #4623 documentation pass. |
me2seeks
left a comment
There was a problem hiding this comment.
Reviewing with real edge cases in mind, grounded in production data from a kimi-coding-plan/k3-256k workspace (see also #4634 — this PR fixes its root causes 1 and 2; items 3 and 4 there remain open).
Overall the direction is sound: every trigger is now a provider-reported number, and I live-verified the new summarizer request shape against k3-256k (details inline on the summarizer change). My concerns are about the paths where the provider never speaks: silent over-window acceptance, output-limit cuts caused by our own maxOutputTokens, and unrecognized error shapes.
| ); | ||
| return values.length > 0 ? Math.min(...values) : undefined; | ||
| } | ||
| return relayModelProfile(connection, modelId)?.contextWindow; |
There was a problem hiding this comment.
Real edge case from live data: kimi-coding-plan/k3-256k accepts requests beyond its fetched window without rejecting or truncating — observed a session accepted at 322K input tokens against a fetched contextWindow of 262,144, with usage still growing monotonically (305K → 322K). On such a provider every signal this design reads stays dark for an undeclared window:
- no rejection → reactive fold never fires;
- usage never plateaus →
context_provider_droppingnever fires (and it is mid-turn only anyway); context_window_suggestiononly fires on a surfaced rejection → never written;- a fetched
/modelswindow is not a declaration → the proactive trigger below never arms.
Net effect: an undeclared kimi session degrades silently and indefinitely (this is exactly the data behind #4634). I agree with "Maka does not decide for the user" as a principle, but for providers observed to accept over-window requests, could the fetched window seed a default declaration (user-overridable), or should the cross-turn plateau check (#4623 PR D) be a blocker rather than a follow-up?
| } | ||
| // Fail closed: reset on every step boundary so a missing final | ||
| // step's usage does not leave a stale value from an earlier step. | ||
| lastStepInputTokens = stepUsage?.inputTokens; |
There was a problem hiding this comment.
pendingLengthFold is set on any length finish with no size or cause guard. Two real cases where the fold is useless or harmful:
- The cut came from our own configured
maxOutputTokens, not the window: maka sendsmaxOutputTokens: 131072on kimi-coding-plan main requests (fromlookupModelMetadata). A reply cut at that cap is not helped by folding history at all — the next request carries the same cap. If the model keeps wanting more output (long reasoning at effort=max, large file generation), this turn folds every step, repeatedly shrinking history while never addressing the constraint. - A user who declares a small output limit for cost control gets routine
lengthfinishes → folds at a few K tokens of context, paying a summarizer call each time.
Suggest gating the length fold on baseline + replyReserve >= capacity (i.e. plausibly window-caused), or capping length-folds per turn.
| const overWindow = | ||
| state.capacity !== undefined && | ||
| state.baselineTokens !== undefined && | ||
| state.baselineTokens + state.replyReserveTokens >= state.capacity; |
There was a problem hiding this comment.
For k3-256k, declaredModelOutputLimit resolves to 131,072 from bundled metadata — half of its 262,144 window. A user who pins the full window gets proactive folding at baseline >= 131,072, i.e. ~50% utilization, which is far earlier than "stay under your declared window" suggests. Reserving the output limit is right in principle (without it a real-size declaration is never crossed), but when the declared output limit is a large fraction of the window the reserve dominates. Consider capping it, e.g. min(outputLimit, capacity / 4) or a fixed bound.
| abortSignal, | ||
| } = input; | ||
| const { turnId, state, queue, activeToolsForStep, abortSignal } = input; | ||
| if (state.malformedSummaryFailure) { |
There was a problem hiding this comment.
The per-send latch and the fingerprint circuit still only cover malformed-summary reasons. After a provider_error / output_length fail-open, baselineTokens stays set, so overWindow remains true and every subsequent step retries the summarizer. Observed live on kimi-coding-plan (which reports failures as HTTP 200 + error body): 15 consecutive failed summarizer calls over ~47 minutes, each 30–45 s, before the main request. Suggest latching any summarizer failure for the rest of the send, with the same fail-open diagnostic. (#4634 item 3)
| // an OpenAI-compatible relay or a local Ollama returns none, and the | ||
| // proactive compaction baseline, the eviction check, and the usage | ||
| // indicator all go dark for exactly the connections that need them. | ||
| includeUsage: adapter.includeUsage ?? true, |
There was a problem hiding this comment.
Defaulting includeUsage on for every non-opted-out OpenAI-compatible server is the right call for usage coverage, but a strict relay/proxy that rejects unknown fields (older vLLM, some gateways) will 400 every streaming request after upgrade, and opting out requires a registry code change. Consider a one-time fallback: on a 400 whose error text mentions stream_options/include_usage, retry without it and remember that for the connection.
| // request. Not-greater, not strictly-fewer: a provider that | ||
| // truncates to a fixed window (Ollama's `num_ctx`) reports the | ||
| // same total on every later request while Maka keeps | ||
| // appending, so a plateau is the signal, and an equal count |
There was a problem hiding this comment.
Two false-positive paths for the dropping note:
- Finalization steps remove all tool schemas (
resolveDispatchreturns[]for the child-summary / sandbox-boundary finalization step), which legitimately drops the next request's input by several K tokens with no fold/prune/image-omit — this condition doesn't exclude that case, so a routine turn end can report "provider is dropping context". - No hysteresis: a 1-token decrease fires the note. Cache accounting or tokenizer drift between calls can produce tiny legitimate decreases. A small relative threshold (e.g. drop > 1–2% or > N tokens) would keep the note meaningful.
| 'Keep each section concise. Preserve exact file paths, function names, commands, and error messages.', | ||
| ].join('\n'); | ||
|
|
||
| const SUMMARY_REQUEST_INSTRUCTION = |
There was a problem hiding this comment.
Confirming this fixes the kimi-coding-plan failure mode from #4634 — I live-tested this exact request shape (system template + trailing user instruction) against k3-256k on both wires: it now returns a fully compliant sectioned summary, where the system-prompt-only shape returned content: null (7 tokens) or continued the conversation as the assistant.
One follow-up worth noting (not blocking): the summarizer request still opens with a different system prompt than the main loop, so it shares no prefix cache with it — the first fold of a session pays a full uncached pass over the folded span (observed 39–88 s on 155–253K inputs). Moving the whole instruction into the trailing user message would let the summarizer reuse the main loop's cached prefix (~99% hits); keeping the template in system is also fine if that's load-bearing for other providers, but the cache cost is real at high context.
|
Thank you for the production data. Every path you named is real, and four of them are fixed on Summarizer retry storm (#4634 item 3). Fixed. The per-send latch recorded only malformed reasons, and the baseline that fired the trigger survives a fail-open, so a
Dropping-note false positives. The finalization case is fixed: the note is suppressed when the step's own active tool set shrank, which is Maka reshaping the request rather than the provider evicting anything. Regression: "does not report provider dropping when the step dropped its tool schemas". On hysteresis I would rather not add a threshold, and I think the
Providers that accept beyond their fetched window. This one I have not changed, and I want to be explicit about why, because you are right that every signal stays dark. A cross-turn plateau check (PR D on #4623) does not rescue it either: your session's usage grows monotonically to 322K, so there is no plateau to detect. The only thing that arms anything is a declaration. Two options I can see:
My preference is 1. Say which you would rather review and I will follow. Prefix cache on the summarizer request. Noted as a follow-up, not forgotten: your 39–88 s uncached first attempts are a strong argument for moving the whole instruction into the trailing user message now that the trailing message exists. I have left the system template in place here because it is load-bearing for the providers the current tests cover, and changing both the shape and the cache behaviour in the same PR would make the k3-256k regression hard to attribute. It belongs on #4634. Thanks also for live-testing the summarizer request shape; that is the part of this PR I could only verify against Ollama and fake providers. |
1faabac to
7759047
Compare
Review findings from apache#4574, each with a test that fails on the previous commit: - The proactive trigger now reserves the model's declared output limit (`connection.models[].maxOutputTokens` or generated metadata, a provider fact; 0 when none is declared) and fires on `>=`: with the window declared at the provider's real size, an accepted request alone can never exceed it, and the reply the next request must leave room for is what tips it. The window itself stays the user's declaration. - The summarizer's provider is a decider again: a context-length rejection from the summary call (or the Codex compaction call) surfaces as `input_too_large`, the reason the planner retreats on. Removing the estimated pre-fit had left that path without a producer. - The eviction check compares input against input: the previous reply's reasoning is not resent on every wire, so input + output is not a floor. - The window suggestion is written after recovery, only when the turn surfaces the rejection, and names the last accepted total, which a fold does not clear. It is no longer a number on the `>` boundary. - The size floor judges an initial fold only; on a roll-forward the summarizer's input is the increment, not the covered span. - `CheckpointSummaryFoldContext` carries usage only; the builder and copy seams validate structure and truncation, which is all they can. - The CLI transcript renders the two new note kinds. - Restored coverage: a synthetic /compact usage row does not shadow the real anchor; the run-header check is exercised with an anchor above the window. - CHANGELOG carries the downgrade and epoch sentences again. The Host's reported `contextWindow` stays the model's window: it is a protocol fact clients already display, and the declared window has its own surface in the composer indicator. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
|
Follow-up on the two open items from your review, both now on The reply reserve is no longer the model's output limit. Your k3-256k arithmetic was the deciding case: reserving 131,072 against a 262,144 declaration folds at half the window a user deliberately chose. It is now twice the last reply the model actually wrote, bounded at 8,000 tokens. Both halves are provider-reported, the margin tracks the session's own behaviour rather than a catalog maximum, and no previous reply means no reserve at all. On your numbers a session answering in 2–3K tokens now folds at roughly 256K of 262K instead of 131K. The old test that asserted the output limit is rewritten around the new rule (a 900-token anchor with a 60-token reply folds against a 1,000 window, the same anchor with a 5-token reply does not, and the model's 600-token output limit is irrelevant to both). A reserve measured from a smaller previous reply can still be too small, so that case is now reported rather than silent: when an accepted request's own input plus its reply exceed the declared window, the transcript carries one That makes three note kinds in this PR, all in the same epoch move (now 103 → 104 after the latest rebase). Still yours to choose: the undeclared-window case on a provider that accepts past its reported window. My preference remains a note when accepted usage exceeds the model's reported window while nothing is declared, rather than seeding a declaration from a fetched number. Say which you would rather review. |
7759047 to
0d0a32d
Compare
Review findings from apache#4574, each with a test that fails on the previous commit: - The proactive trigger now reserves the model's declared output limit (`connection.models[].maxOutputTokens` or generated metadata, a provider fact; 0 when none is declared) and fires on `>=`: with the window declared at the provider's real size, an accepted request alone can never exceed it, and the reply the next request must leave room for is what tips it. The window itself stays the user's declaration. - The summarizer's provider is a decider again: a context-length rejection from the summary call (or the Codex compaction call) surfaces as `input_too_large`, the reason the planner retreats on. Removing the estimated pre-fit had left that path without a producer. - The eviction check compares input against input: the previous reply's reasoning is not resent on every wire, so input + output is not a floor. - The window suggestion is written after recovery, only when the turn surfaces the rejection, and names the last accepted total, which a fold does not clear. It is no longer a number on the `>` boundary. - The size floor judges an initial fold only; on a roll-forward the summarizer's input is the increment, not the covered span. - `CheckpointSummaryFoldContext` carries usage only; the builder and copy seams validate structure and truncation, which is all they can. - The CLI transcript renders the two new note kinds. - Restored coverage: a synthetic /compact usage row does not shadow the real anchor; the run-header check is exercised with an anchor above the window. - CHANGELOG carries the downgrade and epoch sentences again. The Host's reported `contextWindow` stays the model's window: it is a protocol fact clients already display, and the declared window has its own surface in the composer indicator. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
|
Added, on That is the fourth note kind in this PR, all within the same epoch move (103 → 104 after the latest rebase). Local gates are green; the mid-turn suite is 73/73. |
|
Thanks — verified Undeclared window on a provider that accepts past its reported window: I choose option 1 — a note when accepted usage exceeds the model's reported window while nothing is declared. Two reasons beyond the ones you gave:
One constraint from the k3-256k data: on such providers usage crosses the reported window gradually and keeps growing (305K → 322K), so the note should fire once per session when the crossing is first observed (the accepted total first exceeds the reported window), not per send — otherwise it spams on every step past the line. On hysteresis for the dropping note: your I will take the summarizer prefix-cache item back to #4634 as agreed. |
The runtime no longer estimates whether a request fits a context window.
Every "does it fit" question is answered by a provider: the conversation
model's own context-length rejection is recovered by one compact-and-retry,
and the summarizer's provider answers for compaction input (input_too_large
retreats the fold by half). The chars/4 payload ruler, the signed delta
estimate, the 32,000-token fallback history budget, the quarter-window
reserve, the replacement-not-smaller and prefix-over-budget replay gates,
and the final-request rescue re-entry are removed.
Proactive compaction keeps one trigger: the previous accepted request's
real input plus output tokens, as the provider counted them, compared with
the context window the user declared for the model (a model-facts pin or a
relay profile). A provider's /models report and generated metadata are no
longer a threshold on their own. With no declaration there is no proactive
fold; the provider decides. A reply the provider cut at its output limit
(finishReason length) folds once before the next request.
The persisted last-request anchor becomes { inputTokens, outputTokens };
the retired payloadChars key still decodes so 0.2.0 sessions keep loading.
Summaries are capped at 8,000 output tokens with one shorter retry, and the
too-small-for-fold floor reads the summarizer call's real usage instead of
a chars/4 estimate. Two user-visible notes explain provider-side context
changes: context_provider_dropping (an append-only step whose usage did not
grow) and context_window_suggestion (a rejection at a proven-fit total,
with the number the user can declare).
Refs apache#4559
Refs apache#4458, apache#4486
Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Usage is the only signal the runtime's context handling now reads: the proactive compaction baseline, the eviction check, the window suggestion and the usage indicator all start from the provider's own token counts. A Chat Completions server returns none unless the request carries `stream_options.include_usage`, and the factory only sent that for the two providers whose registry entries opted in. A local Ollama and every plain OpenAI-compatible relay — the connection type behind apache#4458 — therefore produced no usage at all, and everything downstream stayed dark. Live probe against Ollama (qwen2.5:0.5b): before, the finish chunk carried an empty usage object and no token_usage event was emitted; after, every turn reports its input and output and persists the anchor. The default is now on for the OpenAI-compatible Chat adapter; a registry entry can still opt out with `includeUsage: false`. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
… can answer The folded span handed to the summarizer usually ends on an assistant message. A chat-template model given a conversation that already ends with its own turn emits an end-of-sequence token and nothing else — observed live on Ollama with qwen2.5:0.5b and qwen2.5:1.5b: finish `stop`, one output token, empty text, on every compaction attempt. The write gate then fails the fold open, so history compaction never succeeded against a local model. The request now closes with a user instruction to write the structured summary; on the first attempt and on the repair. Live against Ollama (qwen2.5:1.5b, declared 1,500-token window): the fold that previously produced an empty summary now writes a 1,737-character checkpoint after one repair, and the next request's provider-counted input drops from 2,954 to 1,222 tokens. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
The two user-visible compaction notes keyed on the `priorReplay` stage only. Since apache#4486 every new fold happens in the request-projection hook (`activeStep`), so the turn that was actually compacted showed nothing and the note arrived one turn later, when the checkpoint was replayed; a fold that failed open in the hook was never surfaced at all. Live against Ollama a fold succeeded and a fold failed in consecutive turns with no note either time. Both predicates now accept a history-compaction decision from either stage; the once-per-send flags in the backend are unchanged. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Review findings from apache#4574, each with a test that fails on the previous commit: - The proactive trigger now reserves the model's declared output limit (`connection.models[].maxOutputTokens` or generated metadata, a provider fact; 0 when none is declared) and fires on `>=`: with the window declared at the provider's real size, an accepted request alone can never exceed it, and the reply the next request must leave room for is what tips it. The window itself stays the user's declaration. - The summarizer's provider is a decider again: a context-length rejection from the summary call (or the Codex compaction call) surfaces as `input_too_large`, the reason the planner retreats on. Removing the estimated pre-fit had left that path without a producer. - The eviction check compares input against input: the previous reply's reasoning is not resent on every wire, so input + output is not a floor. - The window suggestion is written after recovery, only when the turn surfaces the rejection, and names the last accepted total, which a fold does not clear. It is no longer a number on the `>` boundary. - The size floor judges an initial fold only; on a roll-forward the summarizer's input is the increment, not the covered span. - `CheckpointSummaryFoldContext` carries usage only; the builder and copy seams validate structure and truncation, which is all they can. - The CLI transcript renders the two new note kinds. - Restored coverage: a synthetic /compact usage row does not shadow the real anchor; the run-header check is exercised with an anchor above the window. - CHANGELOG carries the downgrade and epoch sentences again. The Host's reported `contextWindow` stays the model's window: it is a protocol fact clients already display, and the declared window has its own surface in the composer indicator. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
|
Adopted the once-per-crossing constraint on Your two reasons are both in the code's favour: the note is the durable Desktop signal, and it names the reported window, which is the number #4575 offers to apply in one click. Noted on the withdrawn hysteresis and on taking the prefix-cache item to #4634. |
Two new system_note kinds and the reshaped lastRequestAnchor land in @maka/core's closed allowlists. A merge-base client handshakes on strict epoch equality and would then fail to decode the first transcript carrying them; the epoch moves so the pair refuses each other at the handshake (main is at 104, so this lands as 105). Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
…epair The repair request carries a longer system prompt than the first one. When the first summary fit but the repair was rejected for context length, the repair catch filed the rejection under the initial malformed defect, so the planner never saw `input_too_large` and never retreated. Classify it before wrapping. Regression test covers "initial malformed, repair overflow". Also align both architecture documents with the implemented active-turn trigger: previous accepted usage plus the reply reserve (the model's declared `maxOutputTokens`, zero when unknown) reaching the declared window, not bare usage exceeding it. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Every OpenAI-compatible chat request asks for `stream_options.include_usage`. A relay that rejects the field answers 400; the runtime surfaces that as the provider's own error and does not resend without the field. A connection that cannot report usage has no context baseline, and the user should learn that from the error, not from a compaction that never happens. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
The check required a strict decrease, but the copy, the PR summary and apache#4623 all define the signal as "content was appended but usage did not grow", and the live Ollama evidence is a plateau at 3,716 input tokens, not a drop: a provider truncating to its own window reports the same total on every later request. Compare with `<=` so the promised contract is the implemented one. An equal count after an append-only step is already impossible without provider-side eviction or rewriting. Also sync two passages in the Chinese architecture document that still carried the retired size policy: the materialization inputs and the checkpoint's replay question now match the English text. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
…note Three defects found by review against production kimi-coding-plan/k3-256k data (apache#4634), each on a path where the provider does not speak clearly. The per-send failure latch only recorded malformed-summary reasons, but the baseline that fired the trigger survives a fail-open, so after a provider error or an output-length failure every later step re-evaluated the same condition and dispatched the same doomed call: 15 consecutive failed summarizer calls over ~47 minutes before one main request. Latch every fail-open reason for the rest of the send. A `finishReason: length` was folded unconditionally, but a reply that stopped at the `maxOutputTokens` Maka itself sends was cut by Maka's own budget, and the next request carries the same budget. Folding there shrinks history every step without touching the constraint. Only a cut below that budget is the provider running out of room. The provider-dropping note excluded folds, prunes and image omissions but not Maka's own tool-schema changes: a finalization step resolves an empty tool set and legitimately drops several thousand schema tokens, which read as the provider evicting context. Refs apache#4559, apache#4634 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
The reply reserve that arms the proactive threshold was the model's declared maximum output. On a model whose limit is a large fraction of its window (k3-256k reports 131,072 against 262,144) that folds at half the declared window, long before the user's declaration is anywhere near crossed. Measure the room from the reply the model actually wrote instead: twice the last one, bounded at 8,000 tokens. Both numbers are provider-reported, the margin tracks the session's own behaviour, and no previous reply means no reserve. A reserve measured from a smaller previous reply can still be too small, and that is now visible rather than silent: when an accepted request's own input plus its reply exceed the declared window, the transcript carries one `context_window_overrun` note per send naming both numbers. The next request folds anyway, because the baseline now exceeds the window; the note explains why, and tells a user who wants whole replies to raise the declaration. The mid-turn fixtures are retuned around the new reserve: their steps report 100/20, 150/30 and 120/10, so a default window of 190 keeps the first request inside it and crosses on the second, which is the journey they describe. Refs apache#4559, apache#4634 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
…ndow Some providers accept a request larger than the window their own catalog entry reports, without rejecting it and without truncating: review of this PR observed a kimi-coding-plan session accepted at 322K input tokens against a reported 262,144, with usage still growing (apache#4634). With nothing declared, every signal this design reads stays dark there — no rejection to recover from, no plateau to read, and no declaration to arm the proactive threshold — so the session degrades quietly and indefinitely. One note per send now states the two real numbers: what the exchange used and what the model reports. It changes nothing on its own. A reported window is still a hint rather than a declaration, and Maka still declares nothing on the user's behalf; the note is what makes the silent case visible enough for them to decide. With a declaration in place the overrun note already owns this ground, so the two never fire for the same fact. Refs apache#4559, apache#4634 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
36475d5 to
1283c87
Compare
|
Correction on the previous push: a rebase of mine stopped on a conflict and the branch was pushed mid-rebase, so |
…send A provider that accepts past its own reported window keeps accepting as usage grows (305K → 322K observed), so a per-send note would repeat on every step past the line. Fire on the transition instead: the previous accepted total was still inside the reported window and this one is not. The baseline carries that previous total through the persisted anchor, so a resumed session does not repeat a crossing it already reported. Refs apache#4559, apache#4634 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
1283c87 to
a49dc06
Compare
|
Closing this in favour of a smaller series. Not because anything here was wrong: it is green and every review item on it is either fixed or deliberately carried forward. The reason is that the design converged past what this PR's shape can carry, and a reviewer arriving now would have to reconstruct it from four rounds of comments. Two things changed after the last round, both simplifications, and both invalidate part of this PR rather than adding to it:
A third change is larger and belongs on its own review: when a fold is rejected as too large, retreat to the span the last accepted request's input covered, rather than halving the covered range. That boundary was accepted by this exact model on this exact connection, so it is provably within capacity, where halving is a guess. The last reply's own span is folded separately when it exceeds 24,000 tokens. The full design is written up on #4559 as three modules (judge / send / compact) with the call bounds. Where each review finding goes
The series
Thank you all for the four rounds; the findings are carried, not discarded. |
|
The replacement is #4653: the send module, with the two simplifications described above and the same review findings carried. The compaction module's retreat boundary follows in a second PR. |
Runtime decided locally whether the next request would fit, from a
characters-per-token estimate against a context window it manufactured when
the user declared none. The estimate was wrong in both directions — it counted
an image at its base64 length and a reasoning replay not at all — and it was
wired to a terminal outcome, so a session could be ended by a number no
provider had ever seen. The invariant this replaces it with is not that the
runtime computes no local numbers, but that no local number may terminate a
turn: a local number may only trigger a reversible fold, and the verdict is
the provider's.
What decides now:
- The threshold is the previous accepted request's real `inputTokens +
outputTokens` plus the room the next reply needs, against the context window
the user declared. A provider's `/models` report and generated metadata are
hints beside the setting, never thresholds. With no declaration there is no
proactive threshold at all.
- The reply reserve is `min(2 x last reply, 8000)`, measured from the reply the
model actually wrote rather than the largest it could write: on a model whose
output limit is half its window, reserving the limit would fold at half the
declared window.
- The compaction module is entered at most once per send, whatever its outcome;
the summarizer's own failure circuit already latches for the send, so a
second entry would dispatch nothing new. Entering is the budget and only the
budget. A folded projection that is actually selected is a separate fact, and
only that one may support a claim about what a still-rejected request
contains: a fold that fails open leaves the raw history in place.
- Whether a request fits is the provider's answer. A classified context-length
rejection folds once and resends; a rejection after an applied fold is
reported as still too large after compaction; an unclassifiable error is
reported as it came, never guessed to be about size.
- A `finishReason: length` drives nothing. The provider running out of window
room and the provider's own lower output cap are indistinguishable from
outside.
What the user sees. Five `system_note` kinds explain the provider-side cases
that used to be silent: the provider dropping or rewriting context (an
append-only step whose input did not grow), a window worth declaring after a
rejection, an exchange that ran past the declared window, a request accepted
past the window the model itself reports while nothing is declared (once per
crossing), and a request still too large after a fold was applied.
Supporting changes. `token_usage` records persist the last-request anchor as
`{ inputTokens, outputTokens }` and still decode the retired `payloadChars`
key. Every OpenAI-compatible chat request asks for `stream_options.
include_usage`, because usage is the only signal this design reads; a relay
that rejects the field is answered once without it and remembered, so the
connection reports no usage rather than failing every request. The summarizer
request ends with a user instruction the model can answer, caps its output at
8,000 tokens, retries once shorter when cut and once stricter when malformed,
surfaces its provider's context-length rejection as `input_too_large`, and
latches any failure for the rest of the send instead of retrying it on every
step.
**Sessions this build writes do not open in earlier releases:** those decode
`token_usage` against a closed allowlist, so the reshaped `lastRequestAnchor`
fails the record and, with it, the Session. Downgrading needs a copy of the
workspace's `runtime.sqlite` taken before the upgrade. Nothing produces the
`context_budget_exhausted` stop reason any more; sessions that recorded it
still decode and present. The Runtime Host compatibility epoch moves to 106.
Design: apache#4559. Supersedes apache#4574, whose review findings are mapped there.
Refs apache#4559, apache#4458, apache#4486, apache#4634
Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Summary
The runtime no longer estimates whether a request fits a context window. Every "does it fit" question is answered by a provider; every trigger is a real number. This is the remaining step after #4486, which removed the fabricated capacity and the two terminal gates but kept the chars/4 delta estimate, the 32,000-token fallback history budget, and a per-step
JSON.stringifyof the whole request as a payload ruler. Design and discussion: #4559 (this PR), #4458 (history).What decides now
context_overflow), and the user's message stays editable.input_too_largeretreats the fold by half, and the loop exits throughno_safe_completed_spanwhen even the smallest legal span is refused.inputTokens + outputTokens, as the provider counted them, compared with the context window the user declared for the model (a model-facts pin or a relay profile). Reasoning tokens are counted whole on purpose — high is the safe direction for a trigger that can only ask for a compaction.finishReason: length) folds once before the next request.What is gone:
estimateNextRequestTokens,exceedsHighWater,midTurnRequestPayloadChars, the final-request rescue re-entry,maxHistoryEstimatedTokensand its five consumers, the quarter-window / 16,384 reserve, the 32,000 fallback, thereplacement_not_smallerandprefix_over_budgetreplay gates, size-based replay selection, the estimated input fit before summarizer calls, and the chars/4 summary-size floor.Notes for the user (four new
system_notekinds):context_provider_droppingwhen an append-only step's usage did not grow (a provider evicting or rewriting context), andcontext_window_suggestionwhen a rejection lands at a proven-fit total, carrying the number the user can declare.Refs #4559 — the issue closes when the whole series (#4574, #4575, #4576) has landed and the remaining items are triaged there.
Refs #4458, #4486
Follow-up PRs (stacked on this one)
The user-facing surfaces are deliberately split out so this PR stays the atomic runtime change and the UI pieces can be reviewed in parallel. Both are open as drafts on top of this branch and will be rebased and undrafted once this merges:
context_window_suggestionnote from this PR is the runtime-side half of that pointer.lastRequestAnchorthis PR introduces (percentage with a declared window, absolute tokens without one, a dash when the provider reports no usage).Not in this series, defined and tracked on #4623 (the degraded modes when the provider does not speak): PR D applies the provider-dropping rule at the turn boundary, PR E adds a "compact and retry" action for provider rejections the error classifier does not recognise. Neither reopens a path this PR touches.
Behaviour changes to review
context_window_suggestionnote then offers the last accepted total as the value to declare. This is the "Maka does not decide for the user" principle; the trade-off is stated in Context budget: stop estimating fit; the provider decides, the Maka window is a user target #4559./modelsreport or models.dev metadata is no longer a threshold on its own.resolveDeclaredContextWindow(runtime) anddeclaredContextWindow(core, single owner of the rule) count only a facts pin or a relay profile.resolveSelectedModelContextWindowkeeps its old meaning for display andcontextRemaining.token_usage.lastRequestAnchoris now{ inputTokens, outputTokens? }. The retiredpayloadCharskey still decodes, so sessions written by 0.2.0 keep loading; sessions written by this build do not open in 0.2.0 (the closed-allowlist cost already noted inCHANGELOG.md).DEFAULT_HISTORY_COMPACT_MAX_OUTPUT_TOKENS; the summarizer'smaxOutputTokensoption was declared and never passed). Alengthfinish gets one shorter retry, then fails open. The too-small-for-fold floor reads the summarizer call's real usage; a producer that reports none is not floored.[block, everything after the boundary]. No local size gate stands between a materializable fold and dispatch.ContextBudgetDiagnostic.maxHistoryEstimatedTokensis retired (decode-only).HistoryCompactionPolicy.midTurnlosesreserveTokens.buildDefaultContextBudgetPolicydrops itsconnectionparameter.stream_options.include_usage), unless the registry opts a provider out. Found live: only two providers opted in, so a local Ollama and every plain OpenAI-compatible relay — the connection type behind Context budget: an undeclared window fabricates a capacity that kills live turns #4458 — returned no usage, and the baseline, the eviction check and the indicator all stayed dark for them.stop, one output token, empty text), so history compaction never succeeded against a local model. Existing message-shape tests gained the trailinguser:text.>=(review).baseline + outputLimit >= declaredWindow, where the limit isconnection.models[].maxOutputTokensor generated metadata — a provider fact — and 0 when none is declared. Without it a window declared at the provider's real size could never be crossed by an accepted request. The window itself remains the user's declaration (see the reply on the review for why the metadata window stays a hint).input_too_largeagain (review), so the planner's retreat-by-half has a producer; the same classification is applied to the Codex compaction call.On dropping the signed delta
#4486 kept a signed chars/4 delta so step 0 could fold before a large new message or image went out. I dropped it: the delta prices an image at a constant and under-reads CJK by about half, so the round trip it saves is the one it is most likely to misjudge; large text results are already capped by the byte-based tool-result archive before they go out; and removing it takes the whole payload ruler and the per-step
JSON.stringify(messages)with it. Happy to be argued back.Test coverage removed
Thirty-seven tests encoded the estimate, the reserve, the payload ruler, the fit gates, or the pre-turn estimate gate; each is either re-pointed at a real-usage observable or removed with the mechanism. Re-pointed:
the usage baseline is the last request's INPUT tokens→input plus output;compacts over the high water→compacts after provider usage crosses the declared window;keeps user_stop when stopping an oversized pre-turn summary→keeps user_stop when stopping while a usage-triggered fold is summarizing;persists the LAST request as the anchor→persists the last request input and output as the anchor;replays a durable pre_turn checkpoint below the current high water→without a local size gate;rejects a paragraph-sized summary for a large folded span→when usage says it is too small. Removed outright (the mechanism is gone): the threemanual compactHistory … exceeds current limits / makes the full replay largercases, the three reserve-derivation cases,anchors on real provider usage plus a tail char/4 delta,credits a SIGNED negative payload delta,falls back to whole-projection char/4 on cold start,high-water crosses at contextWindow minus reserve,a fold that cannot shrink the real payload is refused,a runaway summary is rejected as replacement_not_smaller,bounds the oldest oversized tool result before dispatch,fails with input_too_large before dispatch,charges the summarization instructions against the input budget,a summary at exactly the floor is accepted under ceil-based token estimates,the capacity verdict measures the steering payload,the anchor a finalization step writes excludes the tool schemas it cleared,dispatches a pre-turn Compaction recipe without projecting historyandkeeps a transiently unavailable automatic Compaction checkpoint recoverable(both exercised the deleted pre-turn gate; memory extraction from a hook-triggered fold stays covered),applies max-history overrides to checkpoint replay validation,accepts a complete checkpoint above legacy block limits when the full replay fits,compacts one oversized prior turn before an unknown-model request,compacts an oversized latest turn when an older turn is also retained,dispatches an oversized prior turn when its summary fails, after trying once,never runs a pointless summarizer on a small-window model,a usage object without usable input tokens falls back to cold start,a last-request anchor is only valid as a complete positive pair,a half-written anchor fails the whole token_usage message decode(replaced by the new-shape and retired-key cases).New coverage: the declared-window rule for pin / relay / reported / metadata sources; baseline = input + output,
>not>=, cleared after a fold, seeded from the previous turn, discarded across a model switch; a/modelswindow is not a declaration;pendingLengthFold; the summarizer output cap, shorten retry, andoutput_lengthafter two cuts; usage-based size floor; retreat by half and its give-up exit; structural replay; both notes, including the non-firing cases (pruned step, step 0, declaration already crossed); the retiredpayloadCharskey decodes and an unknown key still fails.Live verification against a real provider (Ollama)
Driven through the real
AiSdkBackendwithgetAIModelagainst a local Ollama (qwen2.5:0.5b/qwen2.5:1.5b, defaultnum_ctx4096), five turns of ~740 tokens each, the previous turn's ledger and run header fed back as prior context.qwen2.5:1.5b{745,10},{1481,10},{2217,10}persisted; baseline 1,491 at turn 3 is<1,500, no fold. Turn 4: baseline 2,227>1,500 → pre_turn fold; summary malformed twice →failedOpen[malformed_summary_missing_section], request dispatched,end_turn. Turn 5: fold again, repair produces a 1,737-char checkpoint →replaced; the request's provider-counted input drops 2,954 → 1,222.token_usageevent at all (Ollama returned no usage because it was never asked); once usage flowed, the summarizer returned empty text every time. Both fixed in this PR.end_turn. Usage grows 745 → 3,700 and then plateaus at 3,716 for turns 6–8 while ~740 tokens are appended per turn: Ollama truncates silently atnum_ctxwith no error.Known limitation, stated plainly: the silent-truncation plateau above happens between turns, and
context_provider_droppingonly fires within a turn (step ≥ 1, on an input count that did not grow), because at step 0 a user edit can shrink the input legitimately. A provider that truncates instead of rejecting therefore gives an undeclared-window user no signal today; declaring the window is the protection. Detecting the cross-turn plateau without an estimate needs a ledger-shape check on the anchor (did anything before the anchored request change?) — a follow-up, tracked on #4559. Closing that gap is PR D on #4623.Verification
Rebased on
main@148f8eb29(after #4578 landed, so this PR no longer carries the dependency bump); the Runtime Host compatibility epoch moves 104 → 105 for the two new note kinds and the reshaped anchor. Locally:npm --workspace @maka/{core,storage,runtime,runtime-host,mcp,ui} run build,npm run typecheck(all workspaces),npm run lint,npm run format:check,npm run check:asf-headers,node scripts/protocol-epoch-check.mjs --base origin/main— all clean. Targeted suites: 425/425 acrossusage-record-last-request-anchor,context-budget-mid-turn-policy,context-budget-model-facts,history-compaction,history-compact-checkpoint,history-compact-summarizer,mid-turn-capacity-backend,overflow-reactive-recovery,ai-sdk-backend; core suite 760/760;provider-conformance24/24,history-compact-summarizer53/53,execution-model-composition28/28 and the CLIpi-transcriptsuites after the review fixes. Not run locally: the fullnpm testmatrix (the Electron/OpenSSL and owned-Host timing cases fail on this machine independently of the branch) — relying on CI for those.Self-review
declaredContextWindowwas implemented twice (runtime andchat-model-choice) in an earlier draft; consolidated into@maka/core/model-thinkingwith runtime delegating. One owner for the one rule the design turns on.@astryxdesign/core@0.5.0; that artifact is not in this PR.memory-extraction.tsandsession-recap.tskeep their own bounded-request budgets; they are auxiliary calls, not history compaction, and are out of scope here.estimatedTokensBefore/Afteron compaction decisions) still use the chars/4 estimator. They are recorded numbers, not decisions, and are kept so existing telemetry consumers do not break.AI use
Select exactly one:
Tool(s) and scope: Claude Code — design discussion, implementation, tests, and review; verified by the author.
Checklist
Does this PR entail a change in behavior?
https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J