From fc6a5d732370365399298b2e767238b0de0490cd Mon Sep 17 00:00:00 2001 From: zhangqingkun976 <1047045074@QQ.COM> Date: Mon, 21 Sep 2026 12:48:15 +0800 Subject: [PATCH] fix(agent-runtime): calibrate the context estimate against what requests cost Reopens #683 with the four points from the review addressed. 1. Conservative handling of anomalous usage. The correction is asymmetric now: upward applies once three observations exist, while downward needs `CONTEXT_CALIBRATION_DOWNWARD_CONSECUTIVE` agreeing samples, is capped at `CONTEXT_CALIBRATION_DOWNWARD_STEP_MAX` per step, and can never take the value below `CONTEXT_CALIBRATION_FACTOR_MIN` (0.85) of the raw estimate. A projection at `CONTEXT_CALIBRATION_BOUNDARY_GUARD_RATIO` (1.18x) of the hard limit therefore still compacts. A report outside 0.5x-3x of what the calibration predicted is a misreport: it is discarded and moves nothing, and two consecutive ones (`CONTEXT_CALIBRATION_ANOMALY_FREEZE`) freeze the downward direction until a usable report arrives. 2. Ratio and fixed offset are separated. The per-character bias stays a scale-free ratio over the guessed tail. An unanchored residual is applied as a ratio only for observations at a comparable scale (`CONTEXT_CALIBRATION_SCALE_BAND_*`, 0.5x-2x of the estimate); otherwise only the observed overhead is added, capped by `CONTEXT_CALIBRATION_OVERHEAD_CAP`. A 100k-scale sample can no longer be applied as a ratio to a 1M projection, and once the large scale is observed the correction becomes proportional instead of staying at the small-scale offset. 3. Regression coverage for the four areas the review named: consecutive under-reporting (the floor holds; a downward move needs agreeing samples), different context scales (ratio in band, capped overhead out of band, proportional once observed), real CJK text (the tail is corrected to the measured cost, ratio inside the documented 2-4 band), and the compaction boundary (the guard ratio still compacts; the deferral is bounded). 4. Rebased onto current main (79cd7ae8) with `contextBudgetFor` from the budget formula extraction (#733), and the full agent-runtime suite re-run. C:\Users\10470\.pi-desktop\scratch\cb9e2d41-8a55-4a18-899d-92ca2791ab51\commit-683-r.txt --- docs/spec/03-runtime/02-agent-runtime.md | 20 + docs/spec/06-delivery/04-e2e-test-plan.md | 22 + docs/spec/08-meta/decisions-log.md | 19 + .../zh-CN/spec/03-runtime/02-agent-runtime.md | 10 + .../spec/06-delivery/04-e2e-test-plan.md | 11 + docs/zh-CN/spec/08-meta/decisions-log.md | 10 + .../src/context-calibration.test.ts | 244 ++++++++++ .../agent-runtime/src/context-calibration.ts | 430 ++++++++++++++++++ packages/agent-runtime/src/runtime.test.ts | 85 ++++ packages/agent-runtime/src/runtime.ts | 75 ++- 10 files changed, 925 insertions(+), 1 deletion(-) create mode 100644 packages/agent-runtime/src/context-calibration.test.ts create mode 100644 packages/agent-runtime/src/context-calibration.ts diff --git a/docs/spec/03-runtime/02-agent-runtime.md b/docs/spec/03-runtime/02-agent-runtime.md index 6cbca8b4da..7b53180a22 100644 --- a/docs/spec/03-runtime/02-agent-runtime.md +++ b/docs/spec/03-runtime/02-agent-runtime.md @@ -471,6 +471,26 @@ boundary falls, not what survives it. The active-user retention limit is 20,000 tokens, capped at half the hard budget so retention alone cannot fill a small window and leave the summary no room. None of these values are configurable. +**Estimate calibration (D606).** Every threshold above is compared against one +number, and that number is corrected against what requests actually cost. pi's +`estimateContextTokens` anchors on the last assistant usage and estimates +everything after it as `chars / 4`: that constant under-counts CJK text, and +with no anchor left it omits the system prompt and the tool schemas, which the +next request still pays for. The two errors are measured and applied +separately — the per-character bias as a scale-free ratio over the guessed +tail, and an unanchored residual as a ratio only for observations taken at a +comparable scale (0.5×–2× of the estimate), otherwise as the observed overhead +capped at 32,000 tokens. + +The correction is asymmetric because this number gates compaction: upward +applies once three observations exist, downward needs three agreeing samples, +is capped at 15 % per step and can never take the value below 85 % of the raw +estimate, so a projection at 1.18× the hard limit (`1 / 0.85`) still compacts. +A report outside 0.5×–3× of what the calibration predicted is treated as a +misreport, and two consecutive misreports freeze the downward direction until a +usable report arrives. + + The provider request layer also caps the concrete output budget before every parent, subagent, and one-shot request. It estimates the serialized input with the pi-ai chars/4 baseline plus a CJK correction, then reserves the larger of diff --git a/docs/spec/06-delivery/04-e2e-test-plan.md b/docs/spec/06-delivery/04-e2e-test-plan.md index 97391becec..8cf305add4 100644 --- a/docs/spec/06-delivery/04-e2e-test-plan.md +++ b/docs/spec/06-delivery/04-e2e-test-plan.md @@ -7957,6 +7957,7 @@ identify the platform validation still needed. | C — Conversation & stream (delegate context budget) | E2E-SUBAGENT-context-overflow-compacts-before-failing, E2E-SUBAGENT-context-overflow-reports-actionable-failure, E2E-SUBAGENT-resume-seeds-within-context-budget | | Quality (delegate context budget) | E2E-SUBAGENT-context-overflow-compacts-before-failing, E2E-SUBAGENT-context-overflow-reports-actionable-failure, E2E-SUBAGENT-resume-seeds-within-context-budget | | M6+ (delegate context budget) | E2E-SUBAGENT-context-overflow-compacts-before-failing, E2E-SUBAGENT-context-overflow-reports-actionable-failure, E2E-SUBAGENT-resume-seeds-within-context-budget | +| C / F / Quality — The context estimate stays safe (calibration) | E2E-CONTEXT-estimate-calibration-stays-safe | The `US-UI-*` visual scenarios (§UI shell visual scenarios) trace to the Codex parity decisions in [decisions-log §D](../08-meta/decisions-log.md) @@ -14085,6 +14086,27 @@ the latest destination. These assertions measure work counts, not device FPS. success fixture uses a child-only extra CA; it does not reproduce a specific antivirus installation or claim native macOS/Linux verification. +### E2E-CONTEXT-estimate-calibration-stays-safe + +- **Preconditions:** Deterministic provider fixture whose reported usage can be + scripted; a session whose occupancy is close to the hard boundary; no real + provider credentials. +- **Steps:** Report usage below the prediction repeatedly and confirm the + displayed occupancy never falls below the floor and compaction still fires; + report usage far outside the plausible band and confirm the value does not + move; run a CJK-heavy session and compare the displayed occupancy with the + reported usage. +- **Expected:** The calibrated occupancy stays within 0.85×–6× of the raw + estimate, moves up as soon as there is evidence, moves down only after + agreeing samples, and a projection at 1.18× the hard limit still compacts. + Misreports move it in neither direction. CJK text no longer reads at a + quarter of its measured cost. +- **Specs linked:** `03-runtime/02-agent-runtime.md` §5.1, + `08-meta/decisions-log.md` D606 +- **Acceptance:** C (chat and stream), F (persistence), Quality +- **Status:** Unit-covered (`context-calibration.test.ts`, including the CJK and + boundary cases); desktop E2E pending + ### E2E-MCP-HTTP-ACK — HTTP acknowledgement and authorization status - **Preconditions**: A local mock Streamable HTTP server returns JSON for diff --git a/docs/spec/08-meta/decisions-log.md b/docs/spec/08-meta/decisions-log.md index 586981d856..466abeec14 100644 --- a/docs/spec/08-meta/decisions-log.md +++ b/docs/spec/08-meta/decisions-log.md @@ -6470,3 +6470,22 @@ that was sitting at the bottom — including after the turn had finished. - Renderer CSS and the theme surface regression change only. There is no scroll state, protocol, persistence, theme schema, or permission change. See `04-ux/08-component-spec.md` and E2E-CHAT-opaque-floating-decision-and-retry-surfaces. + +## 2026-09-21 — The context estimate is calibrated conservatively (D606) + +- Every budget threshold reads one number, and that number is corrected against + what requests actually cost. pi's estimator anchors on the last assistant + usage and estimates the rest as `chars / 4`, which under-counts CJK text by + roughly a factor of 2–4 and, with no anchor left, omits the system prompt and + tool schemas entirely. +- The two errors are applied separately: the per-character bias as a + scale-free ratio over the guessed tail, and an unanchored residual as a ratio + only for observations taken at a comparable scale (0.5×–2×), otherwise as the + observed overhead capped at 32,000 tokens. A 100k-scale sample therefore + cannot be applied as a ratio to a 1M projection. +- The correction is asymmetric: upward applies once three observations exist; + downward needs three agreeing samples, is capped at 15 % per step, and can + never take the value below 85 % of the raw estimate, so a projection at + 1.18× the hard limit still compacts. A report outside 0.5×–3× of what the + calibration predicted is a misreport, and two consecutive misreports freeze + the downward direction. See `03-runtime/02-agent-runtime.md` §5.1. diff --git a/docs/zh-CN/spec/03-runtime/02-agent-runtime.md b/docs/zh-CN/spec/03-runtime/02-agent-runtime.md index d80bc2e15c..4c2ccb3652 100644 --- a/docs/zh-CN/spec/03-runtime/02-agent-runtime.md +++ b/docs/zh-CN/spec/03-runtime/02-agent-runtime.md @@ -370,6 +370,16 @@ Headroom 是 16,384 个代币储备底线的最大值，模型最大输出 小窗口，没有留下摘要的空间。这些值都不是 可配置。 +**估算校准（D606）。** 上述每个阈值都对着同一个数字比较，而该数字会按请求的真实开销校正。pi 的 `estimateContextTokens` +以最后一条助手用量为锚，其余一律按 `chars / 4` 估算：该常数会低估中文文本，且在没有锚点时完全不含系统提示与工具结构， +而下一次请求仍要为它们付费。两类误差分开处理——逐字符偏差以「猜测尾部」上的比例表示（与量级无关）；无锚点残差只在与之 +量级相当的样本（0.5×–2×）上按比例应用，否则只加上观测到的固定开销（上限 32,000 词元）。 + +校正是不对称的，因为这个数字决定压缩：向上修正只要有三个观测即生效；向下修正需要三个方向一致的样本、每次最多 15 %， +且永远不能把数值压到原始估算的 85 % 以下——因此处在硬限制 1.18×（`1 / 0.85`）的投影仍会触发压缩。偏离校准预测 +0.5×–3× 之外的报告视为误报；连续两次误报会冻结向下修正，直到出现可用报告。 + + 传入的用户提示先于第一个提供商参与预算 请求。自动摘要请求在有界的 pi-ai 重试策略下重试瞬时的提供商失败 （3 次重试，2s/4s/8s 退避，Stop 可取消）；配额、鉴权等确定性失败立即返回。 diff --git a/docs/zh-CN/spec/06-delivery/04-e2e-test-plan.md b/docs/zh-CN/spec/06-delivery/04-e2e-test-plan.md index 5ef8346c60..0ec2af51c9 100644 --- a/docs/zh-CN/spec/06-delivery/04-e2e-test-plan.md +++ b/docs/zh-CN/spec/06-delivery/04-e2e-test-plan.md @@ -5269,6 +5269,7 @@ IPC 请求无法关闭。 | C — 对话和直播（委托上下文预算） | E2E-SUBAGENT-context-overflow-compacts-before-failing、E2E-SUBAGENT-context-overflow-reports-actionable-failure、E2E-SUBAGENT-resume-seeds-within-context-budget | | 品质（委托上下文预算） | E2E-SUBAGENT-context-overflow-compacts-before-failing、E2E-SUBAGENT-context-overflow-reports-actionable-failure、E2E-SUBAGENT-resume-seeds-within-context-budget | | M6+（委托上下文预算） | E2E-SUBAGENT-context-overflow-compacts-before-failing、E2E-SUBAGENT-context-overflow-reports-actionable-failure、E2E-SUBAGENT-resume-seeds-within-context-budget | +| C / F / 品质 —— 上下文估算保持安全（校准） | E2E-CONTEXT-estimate-calibration-stays-safe | `US-UI-*` 视觉场景（§UI shell 视觉场景）追踪到 [决策日志 §D](/zh-CN/spec/08-meta/decisions-log) 中的法典平价决策 @@ -8316,6 +8317,16 @@ the latest destination. These assertions measure work counts, not device FPS. - **规格：** 04-ux/06-settings-ia、04-ux/08-component-spec、 04-ux/09-interaction-patterns；ADR turn-process-and-thinking-display。 +### E2E-CONTEXT-estimate-calibration-stays-safe + +- **先决条件：** 可脚本化上报用量的确定性提供商夹具；占用接近硬边界的会话；不使用真实凭据。 +- **步骤：** 连续上报低于预测的用量，确认显示占用不低于下限且压缩仍然触发；上报远离合理区间的用量，确认数值不动；跑一个 + 中文为主的会话，比较显示占用与上报用量。 +- **预期：** 校准后的占用保持在原始估算的 0.85×–6× 之内；有证据即上调；只有方向一致的样本才下调；处在硬限制 1.18× 的 + 投影仍触发压缩；误报不产生任何方向的移动；中文文本不再只有实测成本的四分之一。 +- **规格：** 03-runtime/02-agent-runtime §5.1、08-meta/decisions-log D606。**验收：** C（对话与流）、F（持久化）、品质。 + **状态：** 单元测试覆盖（含中文与边界用例）；桌面 E2E 待补。 + ### E2E-MCP-HTTP-ACK — HTTP acknowledgement and authorization status - **Preconditions**: A local mock Streamable HTTP server returns JSON for diff --git a/docs/zh-CN/spec/08-meta/decisions-log.md b/docs/zh-CN/spec/08-meta/decisions-log.md index ef92c7d469..83b0babefe 100644 --- a/docs/zh-CN/spec/08-meta/decisions-log.md +++ b/docs/zh-CN/spec/08-meta/decisions-log.md @@ -4599,3 +4599,13 @@ that amendment are retired by ADR 0268; the upstream work-panel lifecycle stays. - 仅改动渲染器 CSS 与主题表面回归测试；不改变滚动状态、协议、持久化、主题 schema 或权限。见 `04-ux/08-component-spec.md` 与 E2E-CHAT-opaque-floating-decision-and-retry-surfaces。 + +## 2026-09-21 —— 上下文估算采用保守校准（D606） + +- 所有预算阈值读取同一个数字，该数字按请求的真实开销校正。pi 的估算器以最后一条助手用量为锚、其余按 `chars / 4` 估算， + 这会低估中文文本约 2–4 倍；没有锚点时还会完全漏掉系统提示与工具结构。 +- 两类误差分开应用：逐字符偏差是按比例（与量级无关）作用于猜测尾部；无锚点残差只在量级相当的样本（0.5×–2×）上按比例 + 应用，否则只加观测到的固定开销（上限 32,000 词元）。因此 100k 量级的样本不会作为比例系数套到 1M 的投影上。 +- 校正不对称：向上修正有三个观测即生效；向下修正需要三个方向一致的样本、每次最多 15 %，且不会低于原始估算的 85 %—— + 处在硬限制 1.18× 的投影仍会压缩。偏离预测 0.5×–3× 的报告视为误报，连续两次误报冻结向下修正。 + 见 `03-runtime/02-agent-runtime.md` §5.1。 diff --git a/packages/agent-runtime/src/context-calibration.test.ts b/packages/agent-runtime/src/context-calibration.test.ts new file mode 100644 index 0000000000..34abca8e48 --- /dev/null +++ b/packages/agent-runtime/src/context-calibration.test.ts @@ -0,0 +1,244 @@ +import { describe, expect, it } from "vitest"; +import { + CONTEXT_CALIBRATION_ANOMALY_FREEZE, + CONTEXT_CALIBRATION_BOUNDARY_GUARD_RATIO, + CONTEXT_CALIBRATION_DOWNWARD_CONSECUTIVE, + CONTEXT_CALIBRATION_FACTOR_MIN, + CONTEXT_CALIBRATION_FACTOR_MAX, + CONTEXT_CALIBRATION_MIN_SAMPLES, + CONTEXT_CALIBRATION_OVERHEAD_CAP, + CONTEXT_CALIBRATION_RATIO_MAX, + CONTEXT_CALIBRATION_WINDOW, + ContextEstimateCalibration, + type ContextCalibration, +} from "./context-calibration.js"; + +/** A projection with no usage left: the estimator guessed at every message. */ +function unanchored(tokens: number): ContextCalibration { + return { + tokens, + usageTokens: 0, + trailingTokens: tokens, + lastUsageIndex: null, + }; +} + +/** A projection anchored on a real usage: only the tail was guessed at. */ +function anchored( + usageTokens: number, + trailingTokens: number, +): ContextCalibration { + return { + tokens: usageTokens + trailingTokens, + usageTokens, + trailingTokens, + lastUsageIndex: 4, + }; +} + +/** One CJK paragraph, so the character count is real text and not a constant. */ +const CJK_PARAGRAPH = + "上下文估计的关键在于尾部：估算器只对最后一条用量之后的消息按字符数除以四来猜，" + + "而中文正文的真实开销接近每字一个词元。连续低报时不能把估计值一路压到危险区间，" + + "否则压缩会被推迟，请求就会在临界点上失败。无锚点时必须把系统提示与工具结构的固定开销" + + "和按比例的中文字符偏差分开处理，跨量级的样本不能当作比例系数直接套用。"; + +describe("ContextEstimateCalibration", () => { + it("returns the raw estimate until a series has enough observations", () => { + const calibration = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_MIN_SAMPLES - 1; i++) { + calibration.recordUnanchored(100_000, 300_000); + calibration.recordAnchored(200_000, 20_000, 260_000); + } + expect(calibration.correct(unanchored(100_000))).toBe(100_000); + expect(calibration.correct(anchored(200_000, 20_000))).toBe(220_000); + }); + + it("scales only the guessed tail when an anchor exists", () => { + const calibration = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_MIN_SAMPLES; i++) { + calibration.recordAnchored(200_000, 20_000, 260_000); + } + expect(calibration.trailingRatio()).toBe(3); + expect(calibration.correct(anchored(200_000, 10_000))).toBe(230_000); + expect(calibration.correct(anchored(200_000, 20_000))).toBe(260_000); + }); + + // ① Conservative handling of anomalous usage. + describe("under-reporting", () => { + it("cannot be carried below the safety floor by consecutive low reports", () => { + const calibration = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_WINDOW; i++) { + calibration.recordUnanchored(1_000_000, 600_000); + } + const corrected = calibration.correct(unanchored(1_000_000)); + expect(corrected).toBeGreaterThanOrEqual( + Math.round(1_000_000 * CONTEXT_CALIBRATION_FACTOR_MIN), + ); + expect(corrected).toBeLessThanOrEqual(1_000_000); + }); + + it("does not move down at all before enough agreeing samples", () => { + const calibration = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_DOWNWARD_CONSECUTIVE - 1; i++) { + calibration.recordUnanchored(1_000_000, 600_000); + } + expect(calibration.correct(unanchored(1_000_000))).toBe(1_000_000); + calibration.recordUnanchored(1_000_000, 600_000); + expect(calibration.correct(unanchored(1_000_000))).toBeLessThan(1_000_000); + }); + + it("moves up as soon as there is evidence", () => { + const calibration = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_MIN_SAMPLES; i++) { + calibration.recordUnanchored(1_000_000, 1_400_000); + } + expect(calibration.correct(unanchored(1_000_000))).toBe(1_400_000); + }); + + it("freezes the downward direction after consecutive misreports", () => { + const calibration = new ContextEstimateCalibration(); + // A corroborated downward streak first, so the freeze is what stops it. + for (let i = 0; i < CONTEXT_CALIBRATION_DOWNWARD_CONSECUTIVE; i++) { + calibration.recordUnanchored(1_000_000, 700_000); + } + expect(calibration.correct(unanchored(1_000_000))).toBeLessThan(1_000_000); + for (let i = 0; i < CONTEXT_CALIBRATION_ANOMALY_FREEZE; i++) { + calibration.recordUnanchored(1_000_000, 200_000); + } + expect(calibration.frozen()).toBe(true); + expect(calibration.correct(unanchored(1_000_000))).toBe(1_000_000); + // A usable report unfreezes it, and an unusable one never counts as + // evidence in the other direction either. + calibration.recordUnanchored(1_000_000, 1_000_000); + expect(calibration.frozen()).toBe(false); + }); + }); + + // ② Ratio versus fixed offset: the same residual, used where it belongs. + describe("scale", () => { + it("applies a ratio only to an estimate of a comparable scale", () => { + const calibration = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_MIN_SAMPLES; i++) { + calibration.recordUnanchored(100_000, 130_000); + } + // A 100k sample says nothing proportional about a 1M projection: only + // the fixed part of the residual is added. + expect(calibration.unanchoredRatio(1_000_000)).toBeNull(); + expect(calibration.overheadTokens()).toBe(30_000); + expect(calibration.correct(unanchored(1_000_000))).toBe(1_030_000); + + // Once the projection is observed at its own scale, the ratio applies. + for (let i = 0; i < CONTEXT_CALIBRATION_MIN_SAMPLES; i++) { + calibration.recordUnanchored(1_000_000, 1_300_000); + } + expect(calibration.unanchoredRatio(1_000_000)).toBeCloseTo(1.3, 5); + expect(calibration.correct(unanchored(1_000_000))).toBe(1_300_000); + }); + + it("caps the fixed part so it stays an overhead", () => { + const calibration = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_MIN_SAMPLES; i++) { + // A 40k session with a 300k report is inside the sanity band but its + // absolute residual is far past a system prompt plus schemas. + calibration.recordUnanchored(40_000, 300_000); + } + expect(calibration.overheadTokens()).toBe( + CONTEXT_CALIBRATION_OVERHEAD_CAP, + ); + }); + + it("keeps a small-context sample from under-correcting a large one forever", () => { + const calibration = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_MIN_SAMPLES; i++) { + calibration.recordUnanchored(100_000, 130_000); + } + const large = 1_000_000; + const first = calibration.correct(unanchored(large)); + // Observed at scale, the correction becomes proportional instead of + // staying at the fixed 30k it measured on the small session. + for (let i = 0; i < CONTEXT_CALIBRATION_MIN_SAMPLES; i++) { + calibration.recordUnanchored(large, 1_400_000); + } + const second = calibration.correct(unanchored(large)); + expect(first).toBe(1_030_000); + expect(second).toBe(1_400_000); + expect(second).toBeGreaterThan(first); + }); + }); + + // ③ Real CJK text: the chars/4 tail is the error this series exists for. + it("corrects a real CJK tail to the measured cost", () => { + const calibration = new ContextEstimateCalibration(); + const cjkChars = [...CJK_PARAGRAPH].length; + const estimatedTail = Math.ceil(cjkChars / 4); + // Measured cost of CJK text in the tokenizers this app talks to: between + // roughly 0.6 and 1.0 tokens per character. + const measuredTail = Math.round(cjkChars * 0.9); + for (let i = 0; i < CONTEXT_CALIBRATION_MIN_SAMPLES; i++) { + calibration.recordAnchored(10_000, estimatedTail, 10_000 + measuredTail); + } + const ratio = calibration.trailingRatio(); + expect(ratio).toBeGreaterThanOrEqual(3); + expect(ratio).toBeLessThanOrEqual(CONTEXT_CALIBRATION_RATIO_MAX); + const correctedTail = + calibration.correct(anchored(10_000, estimatedTail)) - 10_000; + expect(correctedTail).toBeGreaterThanOrEqual(measuredTail * 0.95); + expect(correctedTail).toBeLessThanOrEqual(measuredTail * 1.05); + }); + + // ④ The compaction boundary: a correction may not carry a session past it. + describe("compaction boundary", () => { + it("still compacts a projection inside the guard ratio of the limit", () => { + const calibration = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_WINDOW; i++) { + // The worst case for the boundary: every report says the estimate + // reads high, at the projection's own scale. + calibration.recordUnanchored(950_000, 600_000); + } + const hardLimit = 950_000; + const guarded = Math.round( + hardLimit * CONTEXT_CALIBRATION_BOUNDARY_GUARD_RATIO, + ); + expect(calibration.correct(unanchored(guarded))).toBeGreaterThanOrEqual( + hardLimit, + ); + }); + + it("bounds how far a correction may defer the boundary", () => { + const calibration = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_WINDOW; i++) { + calibration.recordUnanchored(950_000, 400_000); + } + const hardLimit = 950_000; + const corrected = calibration.correct(unanchored(hardLimit)); + const deficit = hardLimit - corrected; + expect(deficit).toBeGreaterThanOrEqual(0); + expect(deficit).toBeLessThanOrEqual( + Math.round(hardLimit * (1 - CONTEXT_CALIBRATION_FACTOR_MIN)), + ); + }); + + + it("caps the correction at the upper band, and rejects a wild report", () => { + const calibration = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_MIN_SAMPLES; i++) { + calibration.recordUnanchored(1_000_000, 2_900_000); + } + const corrected = calibration.correct(unanchored(1_000_000)); + expect(corrected).toBe(2_900_000); + expect(corrected).toBeLessThanOrEqual( + 1_000_000 * CONTEXT_CALIBRATION_FACTOR_MAX, + ); + + // Four times what the calibration predicts is a misreport, not a + // measurement: it moves the value in neither direction. + const wild = new ContextEstimateCalibration(); + for (let i = 0; i < CONTEXT_CALIBRATION_ANOMALY_FREEZE; i++) { + wild.recordUnanchored(1_000_000, 4_000_000); + } + expect(wild.frozen()).toBe(true); + expect(wild.correct(unanchored(1_000_000))).toBe(1_000_000); + }); + }); +}); diff --git a/packages/agent-runtime/src/context-calibration.ts b/packages/agent-runtime/src/context-calibration.ts new file mode 100644 index 0000000000..d9404c6081 --- /dev/null +++ b/packages/agent-runtime/src/context-calibration.ts @@ -0,0 +1,430 @@ +/** + * Context-estimate calibration. + * + * Every budget decision in this runtime reads one number: the context estimate + * (the automatic compaction check, the idle pre-compaction, and the occupancy + * the desktop shows). That number comes from pi's `estimateContextTokens`, + * whose shape is: + * + * last assistant usage (real, when the transcript still carries one) + * + ceil(chars / 4) over every message after it + * + * Two systematic errors follow from that shape: + * + * 1. `chars / 4` is an English-prose constant. Applied to CJK text it + * under-counts by roughly a factor of two to four, and CJK is the bulk of + * what some sessions send. ASCII fixtures never show it. + * 2. A projection with no assistant usage left — right after a compaction, or + * at the start of a session — is estimated end to end, so the system + * prompt and the tool schemas disappear from the number even though the + * next request still pays for them. + * + * The two errors are different quantities, so they are measured and applied + * separately: + * + * - **trailing ratio** — while an anchor exists the estimator is exact up to + * the messages appended after it. `real - usage` is the true size of those + * messages, and dividing it by the estimator's own trailing number is a + * pure measurement of the per-character bias. It is a ratio, so it is + * scale-free: the same correction applies whether the tail is 2k or 200k + * tokens. + * - **unanchored residual** — with no anchor, `real - estimate` carries the + * missing system/tool overhead *and* the same per-character bias. Those two + * are split here rather than added together: a residual ratio is applied + * only when the observation was taken at a comparable scale + * (`CONTEXT_CALIBRATION_SCALE_BAND_*` of the estimate being corrected), + * and otherwise only the fixed part — the observed overhead, capped by + * `CONTEXT_CALIBRATION_OVERHEAD_CAP` — is added. A sample taken in a 100k + * session therefore cannot be applied as a ratio to a 1M projection. + * + * Safety properties, because this number gates compaction: + * + * - **Asymmetric.** Upward correction applies as soon as there is evidence; + * downward correction needs `CONTEXT_CALIBRATION_DOWNWARD_CONSECUTIVE` + * agreeing samples, is capped at `CONTEXT_CALIBRATION_DOWNWARD_STEP_MAX` + * per step, and can never take the corrected value below + * `CONTEXT_CALIBRATION_FACTOR_MIN` of the raw estimate. A projection that + * reads at or above `CONTEXT_CALIBRATION_BOUNDARY_GUARD_RATIO` of the hard + * limit therefore still reaches it after correction — a session cannot be + * carried past the boundary by its own corrections. + * - **Sanity-checked.** A provider report outside + * `CONTEXT_CALIBRATION_SAMPLE_*_RATIO` of what the calibration predicted is + * treated as a misreport: it is discarded and counted, and + * `CONTEXT_CALIBRATION_ANOMALY_FREEZE` consecutive ones freeze the downward + * direction until a usable report arrives. + * - **Bounded windows and medians.** Each series is the median of its last + * `CONTEXT_CALIBRATION_WINDOW` samples, so one wild report cannot move it. + * + * What this file is not + * --------------------- + * It does not estimate tokens, talk to a provider, or persist anything across + * sessions. The runtime owns the observations; this owns the arithmetic. The + * `view()` snapshot is what a caller can log, so a surprising occupancy can be + * explained after the fact instead of guessed at. + */ + +/** Observations needed before a series is trusted; below this the raw value stands. */ +export const CONTEXT_CALIBRATION_MIN_SAMPLES = 3; + +/** Bounded window per series: the median of these, so one outlier cannot move it. */ +export const CONTEXT_CALIBRATION_WINDOW = 16; + +/** Per-character bias may not be calibrated outside this range. Scale-free. */ +export const CONTEXT_CALIBRATION_RATIO_MIN = 0.75; +export const CONTEXT_CALIBRATION_RATIO_MAX = 4; + +/** + * The corrected value may not leave this band around the raw estimate. The + * lower bound is the dangerous direction — a number that reads low postpones + * compaction — so it is a hard floor that no combination of samples can pass, + * while the upper bound stays generous: an unanchored projection can + * legitimately be missing a whole system prompt's worth of overhead. + */ +export const CONTEXT_CALIBRATION_FACTOR_MIN = 0.85; +export const CONTEXT_CALIBRATION_FACTOR_MAX = 6; + +/** A projection at or above this share of the hard limit still compacts. */ +export const CONTEXT_CALIBRATION_BOUNDARY_GUARD_RATIO = + 1 / CONTEXT_CALIBRATION_FACTOR_MIN; + +/** + * A provider report is only usable as a measurement of the *request*. Outside + * this band around what the calibration predicted, it is a misreport (a cached + * call that never carried the context, a summary request, a gateway + * substituting a number) and it is discarded rather than folded in. + */ +export const CONTEXT_CALIBRATION_SAMPLE_MIN_RATIO = 0.5; +export const CONTEXT_CALIBRATION_SAMPLE_MAX_RATIO = 3; + +/** Consecutive misreports that freeze the downward direction. */ +export const CONTEXT_CALIBRATION_ANOMALY_FREEZE = 2; + +/** Agreeing samples required before the corrected value may move down at all. */ +export const CONTEXT_CALIBRATION_DOWNWARD_CONSECUTIVE = 3; + +/** Most one downward step may take off the raw estimate. */ +export const CONTEXT_CALIBRATION_DOWNWARD_STEP_MAX = 0.15; + +/** How far a sample's scale may sit from the estimate before it stops being a ratio. */ +export const CONTEXT_CALIBRATION_SCALE_BAND_MIN = 0.5; +export const CONTEXT_CALIBRATION_SCALE_BAND_MAX = 2; + +/** The fixed part of an unanchored residual may not exceed this many tokens. */ +export const CONTEXT_CALIBRATION_OVERHEAD_CAP = 32_000; + +/** A report this many times the request is a misreport, never a measurement. */ +const CONTEXT_CALIBRATION_REAL_MAX_RATIO = 20; + +/** The displayed band ignores residuals outside this range. */ +const CONTEXT_CALIBRATION_RESIDUAL_MIN = 0.25; +const CONTEXT_CALIBRATION_RESIDUAL_MAX = 4; + +/** A sample counts as agreeing with a downward move when it reads this much high. */ +const CONTEXT_CALIBRATION_DOWNWARD_EVIDENCE = 0.02; + +export type ContextCalibration = { + /** Real tokens the estimator itself anchors on (the last assistant usage). */ + usageTokens: number; + /** Estimated tokens for the messages after that anchor. */ + trailingTokens: number; + /** Index of the anchor, or null when the projection carries no usage at all. */ + lastUsageIndex: number | null; + /** The estimator's own total: anchor + trailing. Structurally this is the + * object `estimateContextTokens` returns, so callers pass it straight in. */ + tokens: number; +}; + +export type CalibrationView = { + anchoredSamples: number; + trailingRatio: number; + unanchoredSamples: number; + /** Samples whose scale allowed the ratio form to be used. */ + unanchoredScaleSamples: number; + unanchoredRatio: number; + overheadTokens: number; + residualSamples: number; + residualLowRatio: number; + residualHighRatio: number; + /** Consecutive misreports currently seen. */ + anomalies: number; + /** True while misreports keep the downward direction frozen. */ + frozen: boolean; + /** Consecutive samples that said the estimate reads high. */ + downwardStreak: number; + rawTokens: number; + correctedTokens: number; +}; + +/** + * The measured accuracy of the estimate, as the spread of real residuals. A + * ratio of 1 means the prediction was exact; `lowRatio` below 1 means the + * estimate read low (the dangerous direction) as recently as the window shows. + */ +export type ContextCalibrationBand = { + samples: number; + lowRatio: number; + highRatio: number; +}; + +/** Median of a window; the window is never empty when this is called. */ +function median(values: readonly number[]): number { + const sorted = [...values].sort((a, b) => a - b); + const middle = Math.floor(sorted.length / 2); + return sorted.length % 2 === 1 + ? sorted[middle] + : (sorted[middle - 1] + sorted[middle]) / 2; +} + +function usableNumber(value: number): boolean { + return Number.isFinite(value) && value > 0; +} + +function clamp(value: number, min: number, max: number): number { + return Math.min(max, Math.max(min, value)); +} + +type UnanchoredSample = { estimate: number; realTotal: number }; + +/** + * Two rolling series, one per error above. Kept apart on purpose: a session + * that has been compacted has both, and mixing them would apply the system + * overhead twice on an anchored projection. + */ +export class ContextEstimateCalibration { + private readonly trailingRatios: number[] = []; + private readonly unanchored: UnanchoredSample[] = []; + /** `real / predicted` per usable observation; the displayed band. */ + private readonly residuals: number[] = []; + /** Consecutive misreports. */ + private anomalies = 0; + /** Consecutive samples that said the estimate reads high. */ + private downwardStreak = 0; + /** The last value `correct()` returned, for the per-step cap. */ + private lastCorrected: number | null = null; + + /** + * Record one request whose estimate was anchored on a known usage. + * + * `trueTrailing` is derived by the caller as `real − usageTokens`: the anchor + * is the estimator's own number for the prefix, so the difference is the + * actual size of the messages the estimator had to guess at. + */ + recordAnchored( + usageTokens: number, + estimatedTrailing: number, + realTotal: number, + ): void { + if (!usableNumber(realTotal) || !usableNumber(usageTokens)) return; + if (realTotal > usageTokens * CONTEXT_CALIBRATION_REAL_MAX_RATIO) return; + const trueTrailing = realTotal - usageTokens; + // A projection with nothing after the anchor carries no information about + // the per-character bias; a negative one means the anchor over-counted, + // which is not this series' error to absorb. + if (!usableNumber(trueTrailing) || !usableNumber(estimatedTrailing)) return; + const predicted = usageTokens + estimatedTrailing * this.trailingRatio(); + this.recordResidual(predicted, realTotal); + this.recordDownwardEvidence(predicted, realTotal); + this.push( + this.trailingRatios, + clamp( + trueTrailing / estimatedTrailing, + CONTEXT_CALIBRATION_RATIO_MIN, + CONTEXT_CALIBRATION_RATIO_MAX, + ), + ); + } + + /** + * Record one request that had no anchor at all: the estimate covers every + * message, so the measurement carries the missing system/tool overhead. + */ + recordUnanchored(estimate: number, realTotal: number): void { + if (!usableNumber(estimate) || !usableNumber(realTotal)) return; + if (realTotal > estimate * CONTEXT_CALIBRATION_REAL_MAX_RATIO) return; + const predicted = this.correctUnanchored(estimate); + this.recordResidual(predicted, realTotal); + this.recordDownwardEvidence(predicted, realTotal); + this.push(this.unanchored, { estimate, realTotal }); + } + + /** + * Correct one raw estimate. Returns the raw value until its series has enough + * observations, so a fresh session behaves exactly as it did before. + */ + correct(raw: ContextCalibration): number { + const rawTokens = Math.max(0, Math.round(raw.tokens)); + const anchored = + raw.lastUsageIndex !== null && usableNumber(raw.usageTokens); + const corrected = anchored + ? // The anchored part is the provider's own number for the prefix and + // already contains the system/tool overhead of that request, so only + // the guessed tail is scaled — the unanchored residual must not be + // added here. + raw.usageTokens + Math.max(0, raw.trailingTokens) * this.trailingRatio() + : this.correctUnanchored(rawTokens); + const value = this.clampCorrection(corrected, rawTokens); + this.lastCorrected = value; + return value; + } + + /** The per-character bias currently believed, or 1 before there is evidence. */ + trailingRatio(): number { + if (this.trailingRatios.length < CONTEXT_CALIBRATION_MIN_SAMPLES) return 1; + return median(this.trailingRatios); + } + + /** + * The residual ratio currently believed at the estimate's own scale, or + * `null` when no sample was taken at a comparable scale. Scale-free by + * construction: only in-band samples reach it. + */ + unanchoredRatio(atTokens: number): number | null { + const inBand = this.inBandSamples(atTokens); + if (inBand.length < CONTEXT_CALIBRATION_MIN_SAMPLES) return null; + return median(inBand.map((sample) => sample.realTotal / sample.estimate)); + } + + /** + * The fixed part of the residual — the system/tool overhead a projection + * without an anchor is missing — capped so it stays an overhead, or 0 before + * there is evidence. + */ + overheadTokens(): number { + if (this.unanchored.length < CONTEXT_CALIBRATION_MIN_SAMPLES) return 0; + const overheads = this.unanchored.map((sample) => + Math.max(0, sample.realTotal - sample.estimate), + ); + return Math.min( + CONTEXT_CALIBRATION_OVERHEAD_CAP, + median(overheads), + ); + } + + /** What the last `correct()` did, for logging and for tests. */ + view(rawTokens: number, correctedTokens: number): CalibrationView { + const band = this.band(); + return { + anchoredSamples: this.trailingRatios.length, + trailingRatio: this.trailingRatio(), + unanchoredSamples: this.unanchored.length, + unanchoredScaleSamples: this.inBandSamples(rawTokens).length, + unanchoredRatio: this.unanchoredRatio(rawTokens) ?? 0, + overheadTokens: this.overheadTokens(), + residualSamples: band.samples, + residualLowRatio: band.lowRatio, + residualHighRatio: band.highRatio, + anomalies: this.anomalies, + frozen: this.frozen(), + downwardStreak: this.downwardStreak, + rawTokens, + correctedTokens, + }; + } + + /** + * How far the corrected number has actually been from what the provider + * reported, as a ratio (`real / predicted`) over the bounded window. This is + * the band the estimate can honestly be displayed with: the spread of real + * residuals, not a fabricated ±. + */ + band(): ContextCalibrationBand { + const samples = this.residuals.length; + if (samples === 0) return { samples: 0, lowRatio: 1, highRatio: 1 }; + return { + samples, + lowRatio: Math.min(...this.residuals), + highRatio: Math.max(...this.residuals), + }; + } + + /** True while consecutive misreports keep the downward direction frozen. */ + frozen(): boolean { + return this.anomalies >= CONTEXT_CALIBRATION_ANOMALY_FREEZE; + } + + /** An unanchored correction: ratio in band, fixed overhead otherwise. */ + private correctUnanchored(rawTokens: number): number { + if (rawTokens <= 0) return Math.max(0, rawTokens); + const ratio = this.unanchoredRatio(rawTokens); + if (ratio !== null) return rawTokens * ratio; + const overhead = this.overheadTokens(); + return overhead > 0 ? rawTokens + overhead : rawTokens; + } + + /** Samples taken at a scale comparable to `atTokens`. */ + private inBandSamples(atTokens: number): UnanchoredSample[] { + if (!usableNumber(atTokens)) return []; + return this.unanchored.filter( + (sample) => + sample.estimate >= atTokens * CONTEXT_CALIBRATION_SCALE_BAND_MIN && + sample.estimate <= atTokens * CONTEXT_CALIBRATION_SCALE_BAND_MAX, + ); + } + + private recordResidual(predicted: number, realTotal: number): void { + if (!usableNumber(predicted) || !usableNumber(realTotal)) return; + const ratio = realTotal / predicted; + if (!Number.isFinite(ratio)) return; + // A report this far from the prediction is a misreport. Counting it keeps + // the downward direction frozen until a usable report arrives; discarding + // it keeps the series honest. Anomalies never move the value up either: + // an unusable report is not evidence in either direction. + if ( + ratio < CONTEXT_CALIBRATION_SAMPLE_MIN_RATIO || + ratio > CONTEXT_CALIBRATION_SAMPLE_MAX_RATIO + ) { + this.anomalies += 1; + return; + } + this.anomalies = 0; + if ( + ratio < CONTEXT_CALIBRATION_RESIDUAL_MIN || + ratio > CONTEXT_CALIBRATION_RESIDUAL_MAX + ) { + return; + } + this.push(this.residuals, ratio); + } + + /** Does this sample say the estimate reads high enough to justify a step down? */ + private recordDownwardEvidence(predicted: number, realTotal: number): void { + if (!usableNumber(predicted) || !usableNumber(realTotal)) return; + if (realTotal < predicted * (1 - CONTEXT_CALIBRATION_DOWNWARD_EVIDENCE)) { + this.downwardStreak += 1; + return; + } + this.downwardStreak = 0; + } + + private push(series: T[], value: T): void { + series.push(value); + if (series.length > CONTEXT_CALIBRATION_WINDOW) series.shift(); + } + + private clampCorrection(corrected: number, rawTokens: number): number { + if (!Number.isFinite(corrected)) return rawTokens; + // A raw estimate of zero carries no scale to clamp against; the correction + // is then whatever was measured, never negative. + if (rawTokens <= 0) return Math.max(0, Math.round(corrected)); + const value = clamp( + corrected, + rawTokens * CONTEXT_CALIBRATION_FACTOR_MIN, + rawTokens * CONTEXT_CALIBRATION_FACTOR_MAX, + ); + if (value >= rawTokens) return Math.round(value); + // A downward move needs corroboration and may not jump. + if (!this.downwardAllowed()) return rawTokens; + const stepFloor = + this.lastCorrected === null + ? value + : this.lastCorrected * (1 - CONTEXT_CALIBRATION_DOWNWARD_STEP_MAX); + return Math.round(Math.max(value, stepFloor)); + } + + private downwardAllowed(): boolean { + return ( + !this.frozen() && + this.downwardStreak >= CONTEXT_CALIBRATION_DOWNWARD_CONSECUTIVE + ); + } +} diff --git a/packages/agent-runtime/src/runtime.test.ts b/packages/agent-runtime/src/runtime.test.ts index 4aaab91485..7bd626dd20 100644 --- a/packages/agent-runtime/src/runtime.test.ts +++ b/packages/agent-runtime/src/runtime.test.ts @@ -8486,4 +8486,89 @@ describe("DesktopAgentRuntime hosted web search rounds (ADR 0297)", () => { ]); await restored.dispose(); }); + +describe("context estimate calibration", () => { + /** A request-level usage report: the shape `usageFromPi` hands the runtime. */ + const report = (inputTokens: number) => ({ + inputTokens, + outputTokens: 1, + cacheReadTokens: 0, + cacheWriteTokens: 0, + totalTokens: inputTokens + 1, + }); + + const messages = [ + { role: "user" as const, content: "x".repeat(4_000), timestamp: 1 }, + ]; + + it("moves the gate by what past requests actually cost", async () => { + const runtime = createRuntime(); + const raw = (runtime as any).contextBudget(messages).tokens; + expect(raw).toBeGreaterThan(0); + + // Two unanchored reports, each costing three times the estimate: CJK text + // against the estimator's `chars / 4` constant. Below the sample threshold + // the gate has to stay exactly where it was. + for (let i = 0; i < 2; i++) { + (runtime as any).inFlightContextEstimate = { + tokens: raw, + usageTokens: 0, + trailingTokens: raw, + lastUsageIndex: null, + }; + (runtime as any).recordContextCalibration(report(raw * 3), false); + } + expect((runtime as any).contextBudget(messages).tokens).toBe(raw); + + (runtime as any).inFlightContextEstimate = { + tokens: raw, + usageTokens: 0, + trailingTokens: raw, + lastUsageIndex: null, + }; + (runtime as any).recordContextCalibration(report(raw * 3), false); + expect((runtime as any).contextBudget(messages).tokens).toBe(raw * 3); + + await runtime.dispose(); + }); + + it("ignores a failed attempt, a missing report, and a consumed park", async () => { + const runtime = createRuntime(); + const raw = (runtime as any).contextBudget(messages).tokens; + const parked = { + tokens: raw, + usageTokens: 0, + trailingTokens: raw, + lastUsageIndex: null, + }; + const overhead = () => (runtime as any).contextCalibration.overheadTokens(); + + // Two usable observations, then four that must not count. If any of them + // did, the series would already be trusted and the overhead non-zero. + for (let i = 0; i < 2; i++) { + (runtime as any).inFlightContextEstimate = { ...parked }; + (runtime as any).recordContextCalibration(report(raw * 3), false); + } + (runtime as any).inFlightContextEstimate = { ...parked }; + // A failed stream never carried the request... + (runtime as any).recordContextCalibration(report(raw * 3), true); + // ...and it still consumed the park, so this report has nothing to pair + // with rather than reusing the failed attempt's estimate. + (runtime as any).recordContextCalibration(report(raw * 3), false); + // A settled attempt whose response carried no usage at all. + (runtime as any).inFlightContextEstimate = { ...parked }; + (runtime as any).recordContextCalibration(undefined, false); + // A report with nothing parked. + (runtime as any).recordContextCalibration(report(raw * 10), false); + expect(overhead()).toBe(0); + + // The third usable observation crosses the threshold: the two above plus + // this one, each measuring `3 * raw - raw`. + (runtime as any).inFlightContextEstimate = { ...parked }; + (runtime as any).recordContextCalibration(report(raw * 3), false); + expect(overhead()).toBe(raw * 2); + + await runtime.dispose(); + }); +}); }); diff --git a/packages/agent-runtime/src/runtime.ts b/packages/agent-runtime/src/runtime.ts index a6d30872d1..2abfb7a216 100644 --- a/packages/agent-runtime/src/runtime.ts +++ b/packages/agent-runtime/src/runtime.ts @@ -9,6 +9,7 @@ import { BACKGROUND_CONTEXT, compact, convertToLlm, + estimateContextTokens, estimateTokens, prepareCompaction, withAbortSignal, @@ -214,6 +215,11 @@ import { withStreamIdleTimeout, } from "./provider-retry.js"; +import { + ContextEstimateCalibration, + type ContextCalibration, +} from "./context-calibration.js"; + import { rebuildNodeNetworkTransport } from "./node-proxy.js"; import { createProviderTransportHealth, @@ -1560,6 +1566,13 @@ export class DesktopAgentRuntime { */ private providerRequestBytes?: number; private providerRequestMessages?: number; + /** + * The estimate that describes the provider attempt in flight, parked by + * `streamFn` and consumed when that attempt settles with a usage report. + * Without the pair there is nothing to measure the estimator against. + */ + private inFlightContextEstimate?: ContextCalibration; + private readonly contextCalibration = new ContextEstimateCalibration(); /** * Transport cause of the last provider attempt that rejected before any * response arrived, captured where the original Error still exists. pi-ai @@ -1797,6 +1810,12 @@ Delegation rules: this.providerRetryHeaders = undefined; this.providerRequestBytes = undefined; this.providerRequestMessages = context.messages?.length; + // Park what this request is expected to cost, so the usage report that + // settles it can be measured against it (`contextBudget` corrects the + // same shape). + this.inFlightContextEstimate = estimateContextTokens( + context.messages ?? [], + ); // A new model request starts a new transport streak: the evidence that // justified a rebuild does not carry into the next request (issue #234). this.providerFetchFailure = undefined; @@ -5718,7 +5737,56 @@ Delegation rules: } private contextBudget(messages: AgentMessage[]): ContextBudget { - return contextBudgetFor(this.model, messages); + const budget = contextBudgetFor(this.model, messages); + // Correct the raw estimate with what past requests actually cost. The + // `chars / 4` tail is biased on CJK text, and a projection with no usage + // anchor is missing the system/tool overhead; below the sample threshold + // `correct()` returns the raw value unchanged, and the downward direction + // is bounded by `CONTEXT_CALIBRATION_FACTOR_MIN`. + return { + ...budget, + tokens: this.contextCalibration.correct(estimateContextTokens(messages)), + }; + } + + /** + * Fold one provider report into the estimate calibration. + * + * Only a completed, non-aborted response is a measurement: a failed stream + * never carried the request, and counting one would teach the estimator from + * a request the provider rejected. The pair is recorded against the estimate + * parked by `streamFn`, which describes the same context. + * + * The measured value is the request side only (`input + cacheRead + + * cacheWrite`): the response's own output is not in the projection the parked + * estimate described, and it becomes part of the *next* request's anchor. + */ + private recordContextCalibration( + usage: MessageUsage | undefined, + unusable: boolean, + ): void { + const estimate = this.inFlightContextEstimate; + this.inFlightContextEstimate = undefined; + if (!estimate || unusable || !usage) return; + const realRequestTokens = + usage.inputTokens + + (usage.cacheReadTokens ?? 0) + + (usage.cacheWriteTokens ?? 0); + if (realRequestTokens <= 0) return; + const anchored = + estimate.lastUsageIndex !== null && estimate.usageTokens > 0; + if (anchored) { + this.contextCalibration.recordAnchored( + estimate.usageTokens, + estimate.trailingTokens, + realRequestTokens, + ); + } else { + this.contextCalibration.recordUnanchored( + Math.max(0, Math.round(estimate.tokens)), + realRequestTokens, + ); + } } private automaticCompactionNeeded( @@ -6880,6 +6948,11 @@ Delegation rules: ); } const usage = usageFromPi((event.message as any).usage as Usage | undefined); + // Measure the estimator against this request: the parked estimate + // describes the same context, and only a settled, non-aborted + // response actually carried the request. Consumed either way, so a + // failed attempt cannot pair with a later usage report. + this.recordContextCalibration(usage, failed || aborted); const hostedSearch = hostedSearchFromMessage({ content: (event.message as any).content, citations: (event.message as any).hostedSearchCitations,