Skip to content

fix(imap): normalise bare LF line endings in a fetched BODY[] literal - #293

Merged
robert-virkus merged 1 commit into
Enough-Software:mainfrom
hobleyd:fix/imap-fetch-body-bare-lf
Sep 28, 2026
Merged

robert-virkus merged 1 commit into
Enough-Software:mainfrom
hobleyd:fix/imap-fetch-body-bare-lf

Conversation

@hobleyd

@hobleyd hobleyd commented Sep 22, 2026

Copy link
Copy Markdown

Problem

Some servers hand back a message with the bare-LF line endings it was stored with. The MIME parser only recognises \r\n\r\n as the header/body separator, so such a body came back empty: the client showed the headers and nothing else.

2.2.1 (#272) added bare-LF tolerance to TextMimeData, but a BODY[] literal arrives as bytes and goes through BinaryMimeData, which that fix does not cover.

Fix

FetchParser._parseBodyFull inserts the missing CR before each bare LF on the raw bytes before constructing BinaryMimeData. Every other byte is left exactly as the server sent it, and the input is returned untouched (no copy) in the common conforming-CRLF case, since this runs over every fetched body.

The bytes deliberately are not decoded to a String to reuse the TextMimeData fix. The only charset that could be assumed at that point is wrong: a message declares its charset per part, in headers that have not been parsed yet. Decoding with Utf8Decoder(allowMalformed: true) turns every byte of a windows-1252 or latin-1 body into U+FFFD before the part's own charset= is read, so Teší ma arrives as Te ma with no way to recover it. (That was our first attempt, and the existing 8-bit tests caught it.)

Tests

Four new tests in fetch_parser_test.dart: a bare-LF body is not lost; a bare-LF windows-1252 body keeps its declared charset; a conforming CRLF message passes through untouched; a lone CR gains no LF. Three of the four fail against main.

Some servers hand back a message with the bare-LF line endings it was
stored with. The MIME parser only recognises CRLF CRLF as the
header/body separator, so such a body came back empty: the client
showed the headers and nothing else.

2.2.1 added the same tolerance to TextMimeData, but a BODY[] literal
arrives as bytes and goes through BinaryMimeData, which was not covered.
The bytes cannot simply be decoded to a String to reuse that fix: the
only charset that could be assumed at this point is wrong, because a
message declares its charset per part in headers that have not been
parsed yet. Decoding with Utf8Decoder(allowMalformed: true) turns every
byte of a windows-1252 or latin-1 body into U+FFFD before the part's own
charset= is ever read, so `Teší ma` would arrive as `Te ma` with no way
to recover it.

FetchParser therefore inserts the missing CR on the raw bytes, leaving
every other byte exactly as the server sent it, and returns the input
untouched (no copy) in the common conforming-CRLF case.
@hobleyd
hobleyd force-pushed the fix/imap-fetch-body-bare-lf branch from b95e10e to 253fa80 Compare September 28, 2026 05:45
@hobleyd

hobleyd commented Sep 28, 2026

Copy link
Copy Markdown
Author

Rebased onto current main (2e83eba). The conflict was not a code conflict: 1034624 ("chore: update docs") rewrote fetch_parser.dart and fetch_parser_test.dart with CRLF line endings, so the LF-based commit clashed on every line. The change itself is unchanged; the files now use CRLF to match main. Full suite still passes locally (443 tests).

@robert-virkus

Copy link
Copy Markdown
Member

Thanks, you (and you're AI) are on a roll here!

@robert-virkus
robert-virkus merged commit 33aa6df into Enough-Software:main Sep 28, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants