Skip to content

gmail: retry messages after body fetch failures - #244

Open
fallintoplace wants to merge 1 commit into
ClickHouse:mainfrom
fallintoplace:fix/gmail-body-fetch-cursor
Open

gmail: retry messages after body fetch failures#244
fallintoplace wants to merge 1 commit into
ClickHouse:mainfrom
fallintoplace:fix/gmail-body-fetch-cursor

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Problem

Gmail search and body retrieval happen separately. When gog gmail get failed or timed out, the source still created a record from the search metadata with an empty body and advanced the shared timestamp cursor. The recovered message was then behind the cursor, leaving only the header-only record permanently stored.

Fix

  • represent an unsuccessful body retrieval separately from a valid empty body
  • skip records whose body could not be retrieved
  • keep the previous cursor when any message in the batch fails
  • continue returning successfully retrieved messages and rely on inbox ID deduplication for unchanged retries

Verification

Regression coverage exercises a partial failure followed by recovery, an exception during initial sync, and a nonzero body-fetch command.

  • .venv/bin/python -m pytest -q tests/test_gmail_source.py tests/test_source_filters.py tests/test_db.py: 134 passed
  • broader suite: 2941 passed

@fallintoplace
fallintoplace force-pushed the fix/gmail-body-fetch-cursor branch from a816203 to 0eb9a7d Compare August 1, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant