Skip to content

Accept the EOL marker some producers count in a stream /Length - #7

Closed
sebgoubier wants to merge 1 commit into
soadzoor:mainfrom
sebgoubier:fix/flate-trailing-eol-marker
Closed

sebgoubier wants to merge 1 commit into
soadzoor:mainfrom
sebgoubier:fix/flate-trailing-eol-marker

Conversation

@sebgoubier

Copy link
Copy Markdown
Contributor

What this fixes

Two unrelated real-world PDFs failed with Malformed FlateDecode stream. Probed at the failure point, the underlying cause is the same in both:

Trailing junk found after the end of the compressed stream
stream A stream B
complete zlib stream 7760 bytes 1835 bytes
already decoded 8321 bytes 4075 bytes
trailing bytes 1 (0x0d) 1 (0x0a)

A single end-of-line byte. PDF 32000-1 7.3.8.1 puts an EOL marker after the stream data and states it shall not be included in the stream length; these producers counted it. DecompressionStream rejects the stream outright, and the fully decoded payload is thrown away with it.

The shape of the fix

The marker is accepted only after re-decoding the stream without it produces exactly the same output length. That preserves what the strict decoder was protecting: a truncated stream cannot be rescued this way, because removing a trailing byte never completes one. The re-decode only runs on the error path, so valid streams cost nothing.

Nothing else is trimmed, and this is deliberate. testFlate already asserts that a trailing NUL is rejected — that case still fails, untouched. So do spaces, tabs, and more than one marker. 7.3.8.1 specifies an EOL marker, not arbitrary padding, and the new cases pin that boundary in both directions:

  • accepted: LF, CR, CRLF
  • still rejected: 0x20, 0x09, LF LF, LF CR, CRLF LF, and the pre-existing NUL

Checks

  • npm run test:file -- scripts/test-native-filter-semantics.mjs — passes
  • npm testtsc --noEmit clean, 36 / 37 fast files pass
  • npm run test:integration — 41 / 41
  • npm run test:unit — 66 / 68
  • both source PDFs now get past this filter (they hit unrelated later limits, reported separately)

The two failing files (test-text-lod-core.mjs, test-room-segment-extractor.mjs) also fail on main without this change; they are Windows-only path failures, unrelated.

Scope

FlateDecode only. Independent of #4, #5 and #6.

Refs #2

PDF 32000-1 7.3.8.1 puts an end-of-line marker after stream data and
excludes it from /Length. Producers that count it anyway leave one
stray byte after an otherwise complete zlib stream, and
DecompressionStream rejects that as trailing junk. The payload it had
already produced was discarded and the whole document failed to render.

Measured on two unrelated real-world PDFs: a complete zlib stream of
7760 bytes followed by a single 0x0d, and one of 1835 bytes followed by
a single 0x0a. Both had decoded fully -- 8321 and 4075 bytes -- before
the trailing byte turned the result into an error.

The marker is now accepted, but only after re-decoding the stream
without it proves the same output length. That keeps the guarantee the
strict decoder was there for: a truncated stream cannot be rescued this
way, since removing a trailing byte never completes it.

Nothing else is trimmed. The existing case for a trailing NUL still
fails, as do spaces, tabs, and more than one marker: 7.3.8.1 specifies
an EOL marker, not arbitrary padding, and the new cases pin that
boundary.

Refs soadzoor#2

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@soadzoor

Copy link
Copy Markdown
Owner

I had GPT to review this, and it found 3 things to fix before merge:

  1. Recovery depends on the decoder delivering all output before failing.
    The retry discards its decoded bytes and requires its length to equal the output already received. With Chromium-style buffering, a trailing-data error can discard queued output. My reproduction succeeds at 65,536 decoded bytes but fails at 65,537—even though removing the LF produces a valid stream. Recovery needs to supply any missing output from the successful retry. PR code, Chromium implementation
  2. It can remove a legitimate checksum byte.
    If the zlib checksum ends in CR and the producer appends LF, the helper assumes both bytes form the extra CRLF marker. It removes part of the checksum, so recovery fails. I reproduced this through compilePage(). Trying removal of LF alone before trying CRLF would address the ambiguity. PR code
  3. Cancellation during recovery becomes a malformed-PDF error.
    The retry catches every error, including cancellation. I reproduced an aborted operation returning invalid-object instead of aborted. Cancellation and resource-limit errors should propagate. PR code

I applied the changes in this PR + the suggested fixes above in a single commit here: e5e3537

I hope it's okay from your end as well!

@soadzoor soadzoor closed this Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants