fix(storage): stop persisting base64 image bytes on thread parts - #7021
Merged
Conversation
Tool results that read an image (screenshots, crops) land their full base64 payload in `thread_message_parts` — 54 MB across ~700 rows in production, and it rides along into every later prompt folded from those parts. Redact at the existing storage choke point, next to the credential stripping: an image block carrying inline bytes becomes a text placeholder, and a base64 data URL inside a string is dropped. Image blocks that only reference object storage are untouched — those are cheap and the UI still needs them.
pedrofrxncx
enabled auto-merge (squash)
September 4, 2026 20:04
Migration 202 applies the same `serializePayload` pass to what is already stored: 700 rows holding 74 MB of base64. Candidates are found in two stages — `pg_column_size` reads the TOAST pointer without detoasting and cuts 1M rows to ~19k, and only those get detoasted for the `LIKE`. Payloads are then fetched by id in batches of 20, since the matching rows do not fit in one result set. A row with nothing to redact serializes byte-identically and is skipped, so the migration is idempotent. No `down` — the bytes are not recoverable.
decocms Bot
pushed a commit
that referenced
this pull request
Sep 4, 2026
PR: #7021 fix(storage): stop persisting base64 image bytes on thread parts Bump type: patch - decocms (apps/api/package.json): 4.332.0 -> 4.332.1 - @decocms/native (apps/native/package.json): 4.332.0 -> 4.332.1 Deploy-Scope: server
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ELEC-244.
What
Thread
thrd_7xfnXnNkBxn4eeDaNZc0M(electrolux) has threetool_resultparts of 130–236 KB each —tool-Readon a.png, with the whole image inlined as{type:"image", source:{type:"base64", data:"iVBORw0…"}}. Across prod: 686 parts, 54 MB of base64 inthread_message_parts.It is written once and never read back as an image, but it is folded into the message history of every subsequent turn on the thread.
Fix
One guard at
serializePayloadinapps/api/src/storage/thread-message-parts.ts— the same choke point that already strips NULs, lone surrogates, URL userinfo and GitHub tokens, and the one no new producer can bypass:source.dataordata) becomes{type:"text", text:"[image omitted]"}— a text block rather than an image block with gutteddata, so a folded message stays a valid content block;data:<mime>;base64,…URL inside any string is replaced with[base64 data omitted];studio-storage:, signed URLs) are left alone — cheap, and the UI needs them.Live streaming is unaffected; the image still reaches the UI during the run, it just is not durable.
Testing
bun test apps/api/src/storage/thread-message-parts.test.ts— 18 pass, three new cases (inline block redacted, reference block preserved, data URL in text redacted).bun run checkclean.Not in this PR
Backfill of the existing 54 MB — that is a prod write, say the word and I will open it separately.
Summary by cubic
Stops persisting base64 image bytes in thread message parts and backfills the ones already stored — up to 236 KB per part that was otherwise folded into the prompt of every later turn.
[image omitted]text placeholder and base64 data URLs inside strings are redacted; image blocks that only reference storage stay intact.Migration
202-redact-base64-thread-partsrewrites existing rows with the sameserializePayloadpass, skipping rows that serialize unchanged, so it is idempotent.downmigration — the image bytes are not recoverable.Written for commit d02a023. Summary will update on new commits.