Skip to content

fix(core): recognize a DB-first note's storage echo by its accepted content - #1671

Merged
phernandez merged 4 commits into
mainfrom
accepted-content-echo-gate
Oct 7, 2026
Merged

phernandez merged 4 commits into
mainfrom
accepted-content-echo-gate

Conversation

@phernandez

Copy link
Copy Markdown
Member

Summary

On production, about 1 in 15 app saves still fully re-indexed its own storage echo. Trace (tenant c4addccf): the materializer finished the S3 PUT at :58.1, the webhook index read entity.checksum at :59.7, and the materializer recorded the new ETag at :00.6. The gate saw the previous ETag and re-read a note the index already held.

  • EntityRepository.get_by_file_paths also returns the note's accepted db_checksum (note_content at the same path).
  • IndexedChecksums.holds_accepted_content: when the storage checksum does not match, an object whose content checksum equals the accepted one is current.
  • A row masked by a pending graph publication (checksum=None) is still read, so the repair path is unchanged; other bytes are read as before.
  • move_orphan_checksum_source -> content_checksum_source: it now serves the move-vacate marker and the accepted echo. Locally both checksum domains are SHA-256, so the storage checksum serves both.

Tests

  • test-int/test_accepted_content_echo_gate.py (real DB and files): the exact window (accepted revision two in storage, entity still recording revision one) is current; a pending publication still reads; someone else's bytes still read. The accepted case fails without the fix.
  • Updated fake rows and expected IndexedChecksums for the fourth column.

Basic Memory Cloud reads the content checksum from the notification's object metadata (bm-file-checksum), so the check adds no storage request for webhook jobs.

🤖 Generated with Claude Code

https://claude.ai/code/session_01T8wjd6HrtSA2LN9ssC4NzF

…ontent

A DB-first write indexes its accepted Markdown before the file is written,
and the materializer records the stored object's checksum only after the
write. On a fast runtime the object's own storage notification can reach
the index gate in between, find the previous storage checksum, and re-read
and re-index a note the index already holds.

The indexed-checksum lookup now also returns the note's accepted content
checksum. When the storage checksum does not match yet, an object whose
content checksum equals the accepted one is current. A row masked by a
pending graph publication is still read, keeping the repair path, and any
other bytes are read as before. move_orphan_checksum_source becomes
content_checksum_source, since it now serves both content-keyed checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8wjd6HrtSA2LN9ssC4NzF
Signed-off-by: phernandez <paul@basicmachines.co>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-07T21:12:15.902766Z ad21358 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

phernandez and others added 2 commits October 7, 2026 14:17
…lumn

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8wjd6HrtSA2LN9ssC4NzF
Signed-off-by: phernandez <paul@basicmachines.co>
…um column

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8wjd6HrtSA2LN9ssC4NzF
Signed-off-by: phernandez <paul@basicmachines.co>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ff026326b0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +51 to +52
if self.checksum is None or content_checksum is None:
return False

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle first-write echoes when the entity checksum is null

When the storage echo belongs to a newly created DB-first note, create_pending_accepted_entity initializes Entity.checksum to None; after graph publication but before materialization records the storage checksum, this produces IndexedChecksums(checksum=None, accepted_content_checksum=<matching hash>). This guard therefore rejects the exact accepted-content match and causes a full read/reindex instead of marking the echo current, leaving the create path exposed to the duplicate indexing/publication work this change is meant to prevent. The new integration test misses this because it first materializes revision one and only simulates an update; distinguish a deliberately masked incomplete projection from a legitimate not-yet-recorded storage checksum and cover the initial-create echo.

AGENTS.md reference: AGENTS.md:L137-L139

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ad21358: the indexed-checksum query applies the incomplete-projection mask to the accepted checksum as well, so a masked row has nothing to match and is still read, and holds_accepted_content no longer requires a storage checksum. New first-write-echo case (entity checksum unset, object holds the accepted content) is current; it fails with the old guard, and repair-still-reads fails if the query stops masking.

…content

A new note's entity has no storage checksum until its first materialization
settles, and the gate treated that unset checksum as a masked, incomplete
row, so a first write's echo was still fully re-indexed. The indexed-checksum
lookup now applies the incomplete-projection mask to the accepted checksum
too, so a masked row has nothing to match and is still read, while an unset
storage checksum on a complete row no longer blocks the content match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T8wjd6HrtSA2LN9ssC4NzF
Signed-off-by: phernandez <paul@basicmachines.co>
@phernandez
phernandez merged commit 6e49c0e into main Oct 7, 2026
34 checks passed
@phernandez
phernandez deleted the accepted-content-echo-gate branch October 7, 2026 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant