Skip to content

fix(memory): restore latest agent snapshot from batched events - #700

Merged
tejaskash merged 3 commits into
mainfrom
fix/read-agent-latest-batched-snapshot
Oct 13, 2026
Merged

tejaskash merged 3 commits into
mainfrom
fix/read-agent-latest-batched-snapshot

Conversation

@jariy17

@jariy17 jariy17 commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #698

Problem

With batch_size > 1, AgentCoreMemorySessionManager buffers SessionAgent snapshots and flushes them as one AGENT event whose payload is ordered oldest first. read_agent read payload[0], the oldest snapshot, so an agent restored by a new session manager lost the changes from its previous invocation (agent.state, conversation manager state, interrupt state).

Fix

  1. read_agent reads payload[-1], the last snapshot written. Sessions already stored by affected versions also restore their latest snapshot.
  2. Reading the last snapshot depends on two ordering guarantees that didn't hold, so this also:
    • prepends agent states re-buffered after a failed flush. Previously extend put them after states buffered during the failed send, so payload[-1] would restore a stale snapshot.
    • flushes buffered messages before agent states when the agent-state buffer fills. Previously only snapshots were flushed, so a process killed before the next flush left a snapshot whose removed_message_count pointed past the saved messages, and the restored history was empty.

Behavior change for stored sessions

Batched sessions written since 1.23.1 never restored interrupt state. After this change, a session whose last invocation ended in an unanswered interrupt restores it, as with batch_size=1, and the next call must pass interruptResponses; a plain prompt raises Strands' TypeError: must resume from interrupt with list of interruptResponse's.

Testing

  • Unit tests: write→read round trip restores the newest snapshot; failed flush keeps a concurrently buffered state last; agent-state auto-flush persists buffered messages first. Each fails without its fix.
  • pytest tests/bedrock_agentcore/memory/integrations/strands/: 399 passed. ruff check / ruff format --check clean.
  • The AgentCoreMemorySessionManager with batch_size > 1 restores agents without the changes made in their previous invocation #698 repro restores model_calls=3 (was None).
  • Scenario runs against an in-memory data plane, batch_size 2/3/5/10 vs 1: agent.state across turns, SlidingWindowConversationManager(window_size=4) over 8 turns (plateaus as with batch_size=1), interrupt resumed from a new agent, async_mode, mid-invocation flushes, flush failure and retry, sessions written by main then restored by this branch, two agents in one session.
  • Checked against the AgentCore Memory service that ListEvents returns payload entries in the order CreateEvent wrote them.

Not in this PR (pre-existing)

  • With batch_size > 1, legacy agent-event migration in read_agent buffers the migrated snapshot but deletes the legacy event immediately; a crash before the flush loses it.
  • An empty/malformed newest AGENT event makes read_agent return None instead of an older valid snapshot (read_session got guards in fix: handle empty payload gracefully #683, read_agent didn't).
  • _flush_messages_only re-buffers failed messages with extend, so the same ordering issue can reorder messages.

jariy17 added 2 commits October 12, 2026 21:44
With batch_size > 1, one AGENT event holds several SessionAgent
snapshots, oldest first. read_agent read payload[0], so restored agents
lost the state saved during their previous invocation.

Fixes #698
…ssages

Restoring payload[-1] relies on the newest snapshot being last and on
messages being persisted no later than the snapshot that indexes them.

- Prepend agent states re-buffered after a failed flush, so states
  buffered during the failed send stay newest.
- Flush buffered messages before agent states when the agent state
  buffer fills, so a crash can't leave a snapshot whose
  removed_message_count points past the saved history.
@github-actions

Copy link
Copy Markdown
Contributor

✅ No Breaking Changes Detected

No public API breaking changes found in this PR.

@github-actions github-actions Bot added size/s PR size: S and removed size/s PR size: S labels Oct 12, 2026
@jariy17
jariy17 deployed to auto-approve October 12, 2026 22:18 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 12, 2026 22:18 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 12, 2026 22:18 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 12, 2026 22:18 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 12, 2026 22:18 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 12, 2026 22:18 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 12, 2026 22:18 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 12, 2026 22:18 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 12, 2026 22:18 — with GitHub Actions Active
@github-actions github-actions Bot added size/s PR size: S and removed size/s PR size: S labels Oct 12, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Oct 12, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 12, 2026
@jariy17
jariy17 marked this pull request as ready for review October 13, 2026 19:53
@jariy17
jariy17 requested a review from a team October 13, 2026 19:53
@github-actions github-actions Bot removed the size/s PR size: S label Oct 13, 2026
@jariy17
jariy17 deployed to auto-approve October 13, 2026 20:07 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 13, 2026 20:07 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 13, 2026 20:07 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 13, 2026 20:07 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 13, 2026 20:07 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 13, 2026 20:07 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 13, 2026 20:07 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 13, 2026 20:07 — with GitHub Actions Active
@jariy17
jariy17 deployed to auto-approve October 13, 2026 20:07 — with GitHub Actions Active
@github-actions github-actions Bot added the size/s PR size: S label Oct 13, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Oct 13, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Oct 13, 2026

@tejaskash tejaskash left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The fix is correct when flushes don't overlap, and the three new tests fail on main and pass here.

One non-blocking note: the two ordering guarantees in the description hold only when flushes don't overlap. With flush_interval_seconds set, the timer thread can flush while the AfterInvocationEvent flush runs on the agent thread, and both of these still restore a stale snapshot:

  1. A timer flush of s1 fails slowly. Meanwhile the agent thread buffers s2 and flushes it successfully. The prepend then puts s1 back into an empty buffer, the next flush writes it as the newest AGENT event, and read_agent restores s1.
  2. _flush_messages reads the message buffer first and the agent-state buffer later. If the agent thread buffers m2 and s2 in between, s2 is saved and m2 is not.

I reproduced both on this branch and on main, so they don't block this PR. I'll open a separate PR for them.

@tejaskash
tejaskash merged commit 45260c8 into main Oct 13, 2026
45 checks passed

This branch was successfully deployed

1 active deployment
auto-approve — 50aab6b7 Deployed Oct 13, 2026 by jariy17 via Test (services) #1630
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s PR size: S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AgentCoreMemorySessionManager with batch_size > 1 restores agents without the changes made in their previous invocation

2 participants