Repository navigation
cli: scale the checkpoint interval with the transcript - #1574
Conversation
|
|
fcf548f to
4e3a161
Compare
|
This branch currently sits on top of #1575 ( |
85f6773 to
9ba3af2
Compare
…are messages The snapshot timer was a fixed 5 s. A host that knows its transcript is large cannot slow it down, so checkpointing stays at its most expensive setting exactly where it costs the most. Add RunOptions.stateSnapshotIntervalMs, clamped to [1 s, 60 s] so a host cannot disable checkpointing with 0 or Infinity, and use it for the timer. Also replace the deep clone behind onStateSnapshot with copyAgentStateSharingMessages: every container the runtime mutates in place is copied, the message objects are shared. That is what lets a host recognise an extension of the previous snapshot by pointer identity instead of re-serialising the whole history on every snapshot.
In-flight state checkpoints were written on the SDK's fixed 5 s cadence no matter how large the chat had grown, so the largest transcripts paid the most expensive setting: two multi-MB writes and two fsyncs every five seconds. Size the interval from the persisted transcript, read once per run: 5 s under 8 MB, then 10 s / 15 s / 20 s at 8 MB / 16 MB / 24 MB, capped at 20 s. The cost of the larger window is stated rather than hidden: a crash can lose up to 20 s of run state instead of 5 s, and 20 s is reached at 24 MB, not at 12 MB. Depends on the SDK option `RunOptions.stateSnapshotIntervalMs`, so this is based on the SDK PR; without it the field is dropped and the interval stays 5 s.
9ba3af2 to
e502372
Compare
Why
In-flight state checkpoints were written on a fixed 5 s cadence no matter how large the chat had grown, so the largest transcripts paid the most expensive setting: a checkpoint is two multi-MB serializations and two fsyncs, and it ran every five seconds until the run ended.
What changes
utils/run-state-storage.tsgains the interval rule and a single reader for it:checkpointIntervalMsForStateBytesis the pure rule (5 s base, +5 s per 8 MB, capped at 20 s);checkpointIntervalMsForChatstats the chat'srun-state.jsonandchat-messages.jsononce per run and applies it.use-send-message.tspasses the result to the SDK asstateSnapshotIntervalMs, andcreate-run-configforwards it.Based on #1571 (
sdk/state-snapshot-interval): the option is the SDK's. Without that PR this change is inert — the field would be dropped and the interval would stay 5 s.Cost that must be stated
A crash can now lose up to 20 s of run state instead of 5 s, and 20 s is first reached at 24 MB of persisted state, not at 12 MB. Below 8 MB nothing changes.
Verification
mainis red for an unrelated reason (sdk/src/impl/llm.tsprovider options), so the typecheck line is a diff of the error output before and after, not an exit-0 claim.The new tests cover the boundaries of the table (0, just under 8 MB, exactly 8 MB, 24 MB and above), that a missing chat directory falls back to the base interval, and that the value reaches the run config.
Risk and rollback
Only the checkpoint cadence changes; every write is still atomic and still happens on completion. Rollback: revert the commit and the SDK falls back to its own default.
Base
Sits on #1575 (the build fix) and #1577 (CI now runs the suites). Rebased onto that pair, so this PR's checks run
bun testandtscfor real instead of only building the tree and smoke-testing the binary.