Repository navigation
sdk: host-configurable state snapshot interval, and snapshots that share messages - #1571
Conversation
|
|
0b0d760 to
c18b315
Compare
|
This branch currently sits on top of #1575 ( |
Memory, measuredBefore/after on the snapshot path this PR replaces,
Semantics checked, not assumed: the JSON clone returns message objects that are not the live ones ( Why it matters for memory rather than only time: at the 5 s cadence a checkpoint used to allocate ~4.3x the transcript size and ~113 MB of RSS, then hand it to the GC. That churn is what shows as RSS growth and as GC pauses on the host thread; now a snapshot allocates nothing proportional to the history. Scope: this measures the snapshot path, not end-to-end process RSS of a whole CLI session — that would need a replay with and without the patch, which I did not run. |
c18b315 to
d6c0055
Compare
…are messages The snapshot timer was a fixed 5 s. A host that knows its transcript is large cannot slow it down, so checkpointing stays at its most expensive setting exactly where it costs the most. Add RunOptions.stateSnapshotIntervalMs, clamped to [1 s, 60 s] so a host cannot disable checkpointing with 0 or Infinity, and use it for the timer. Also replace the deep clone behind onStateSnapshot with copyAgentStateSharingMessages: every container the runtime mutates in place is copied, the message objects are shared. That is what lets a host recognise an extension of the previous snapshot by pointer identity instead of re-serialising the whole history on every snapshot.
d6c0055 to
153d1ba
Compare
Why
The SDK's state snapshot timer was a fixed 5 s, and
onStateSnapshotdeep-cloned the whole session state, every five seconds, to hand it over. A host that knows its transcript is large could not slow that down — so checkpointing stayed at its most expensive setting exactly where it costs the most.What changes
RunOptions.stateSnapshotIntervalMs, clamped to[1 s, 60 s]so a host cannot disable checkpointing with0orInfinity, used for the snapshot timer instead of the constant.onStateSnapshotnow callscopyAgentStateSharingMessages: every container the runtime mutates in place is copied, the message objects are shared. That is what lets a host recognise an extension of the previous snapshot by pointer identity instead of re-serialising the whole history on each snapshot.A follow-up CLI change (#1574) chooses the interval from the transcript size; it is based on this PR.
Verification
mainis red for an unrelated reason (7 errors insdk/src/impl/llm.ts, provider options typing), so the typecheck line is a diff of the error output before and after, not an exit-0 claim.The two new test files cover the clamp boundaries (
0,Infinity, below 1 s, above 60 s), that the timer uses the host value, and that a snapshot shares message objects while copying every mutated container.Risk and rollback
copyAgentStateSharingMessages.run.ts; the option disappears and the deep clone returns.Base
Sits on #1575 (the build fix) and #1577 (CI now runs the suites). Rebased onto that pair, so this PR's checks run
bun testandtscfor real instead of only building the tree and smoke-testing the binary.