Skip to content

sdk: host-configurable state snapshot interval, and snapshots that share messages - #1571

Merged
obadakhalili merged 1 commit into
CodebuffAI:mainfrom
Salhi-zh:sdk/state-snapshot-interval
Oct 30, 2026
Merged

obadakhalili merged 1 commit into
CodebuffAI:mainfrom
Salhi-zh:sdk/state-snapshot-interval

Conversation

@Salhi-zh

@Salhi-zh Salhi-zh commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Why

The SDK's state snapshot timer was a fixed 5 s, and onStateSnapshot deep-cloned the whole session state, every five seconds, to hand it over. A host that knows its transcript is large could not slow that down — so checkpointing stayed at its most expensive setting exactly where it costs the most.

What changes

  • RunOptions.stateSnapshotIntervalMs, clamped to [1 s, 60 s] so a host cannot disable checkpointing with 0 or Infinity, used for the snapshot timer instead of the constant.
  • onStateSnapshot now calls copyAgentStateSharingMessages: every container the runtime mutates in place is copied, the message objects are shared. That is what lets a host recognise an extension of the previous snapshot by pointer identity instead of re-serialising the whole history on each snapshot.

A follow-up CLI change (#1574) chooses the interval from the transcript size; it is based on this PR.

Verification

cd sdk && bun run typecheck                                              → identical error set to main
bun test src/__tests__/state-snapshot-interval.test.ts \
         src/__tests__/snapshot-session-state.test.ts \
         src/__tests__/byok-run.test.ts \
         src/__tests__/cancelled-session-state.test.ts                   → 30 pass, 0 fail

main is red for an unrelated reason (7 errors in sdk/src/impl/llm.ts, provider options typing), so the typecheck line is a diff of the error output before and after, not an exit-0 claim.

The two new test files cover the clamp boundaries (0, Infinity, below 1 s, above 60 s), that the timer uses the host value, and that a snapshot shares message objects while copying every mutated container.

Risk and rollback

  • A host can now stretch the checkpoint interval to 60 s, which widens the window of run state a crash can lose. The clamp is what keeps that bounded, and the default stays 5 s.
  • Sharing message objects across snapshots means a consumer must not mutate messages from a snapshot in place. All in-repo consumers are readers; the contract is stated on copyAgentStateSharingMessages.
  • Rollback: revert run.ts; the option disappears and the deep clone returns.

Base

Sits on #1575 (the build fix) and #1577 (CI now runs the suites). Rebased onto that pair, so this PR's checks run bun test and tsc for real instead of only building the tree and smoke-testing the binary.

@Salhi-zh

Copy link
Copy Markdown
Contributor Author

build-and-test fails on this PR and would fail on any PR right now: the failure is in Build SDK, in sdk/src/impl/llm.ts, common/src/util/messages.ts and packages/agent-runtime/src/tool-stream-parser.ts — files this PR does not touch. Every recent run on main is a failure for the same reason (689c945, a3b07ad, b986a2b, b8bc600, 1dfca82). Locally, on clean main (a3b07ad), cd cli && bun run typecheck reports 23 errors and cd sdk 7, none in the files changed here; the PR body quotes that before/after diff rather than an exit-0 claim.

@Salhi-zh

Copy link
Copy Markdown
Contributor Author

This branch currently sits on top of #1575 (build/green-main), so the diff shows its four files as well. Reason: main fails Build SDK on every commit, and the check runs before any test, so no PR can be green without that fix. Once #1575 lands I will rebase this onto main and that commit drops out of the diff; nothing in this PR depends on it.

@Salhi-zh

Copy link
Copy Markdown
Contributor Author

Memory, measured

Before/after on the snapshot path this PR replaces, Bun.gc(true) between runs, 30,000 messages / 22.4 MB of session state:

time per snapshot heap allocated RSS delta
previous JSON round-trip clone 70.1 ms 95.6 MB 113.1 MB
snapshotSessionState (this PR) 0.2 ms 0 MB 0 MB

Semantics checked, not assumed: the JSON clone returns message objects that are not the live ones (history[0] !== messages[0]), share-mode returns the same objects (===), and every mutable container (messageHistory, agentContext logs, childRunIds, ancestorRunIds, subagents) is a fresh copy.

Why it matters for memory rather than only time: at the 5 s cadence a checkpoint used to allocate ~4.3x the transcript size and ~113 MB of RSS, then hand it to the GC. That churn is what shows as RSS growth and as GC pauses on the host thread; now a snapshot allocates nothing proportional to the history.

Scope: this measures the snapshot path, not end-to-end process RSS of a whole CLI session — that would need a replay with and without the patch, which I did not run.

@Salhi-zh
Salhi-zh force-pushed the sdk/state-snapshot-interval branch from c18b315 to d6c0055 Compare October 29, 2026 21:22
…are messages

The snapshot timer was a fixed 5 s. A host that knows its transcript is large
cannot slow it down, so checkpointing stays at its most expensive setting
exactly where it costs the most.

Add RunOptions.stateSnapshotIntervalMs, clamped to [1 s, 60 s] so a host cannot
disable checkpointing with 0 or Infinity, and use it for the timer.

Also replace the deep clone behind onStateSnapshot with
copyAgentStateSharingMessages: every container the runtime mutates in place is
copied, the message objects are shared. That is what lets a host recognise an
extension of the previous snapshot by pointer identity instead of
re-serialising the whole history on every snapshot.
@Salhi-zh
Salhi-zh force-pushed the sdk/state-snapshot-interval branch from d6c0055 to 153d1ba Compare October 30, 2026 13:23
@obadakhalili
obadakhalili merged commit 3cfa436 into CodebuffAI:main Oct 30, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants