Accept the EOL marker some producers count in a stream /Length - #7
Closed
sebgoubier wants to merge 1 commit into
Closed
sebgoubier wants to merge 1 commit into
sebgoubier wants to merge 1 commit into
Conversation
PDF 32000-1 7.3.8.1 puts an end-of-line marker after stream data and excludes it from /Length. Producers that count it anyway leave one stray byte after an otherwise complete zlib stream, and DecompressionStream rejects that as trailing junk. The payload it had already produced was discarded and the whole document failed to render. Measured on two unrelated real-world PDFs: a complete zlib stream of 7760 bytes followed by a single 0x0d, and one of 1835 bytes followed by a single 0x0a. Both had decoded fully -- 8321 and 4075 bytes -- before the trailing byte turned the result into an error. The marker is now accepted, but only after re-decoding the stream without it proves the same output length. That keeps the guarantee the strict decoder was there for: a truncated stream cannot be rescued this way, since removing a trailing byte never completes it. Nothing else is trimmed. The existing case for a trailing NUL still fails, as do spaces, tabs, and more than one marker: 7.3.8.1 specifies an EOL marker, not arbitrary padding, and the new cases pin that boundary. Refs soadzoor#2 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Owner
|
I had GPT to review this, and it found 3 things to fix before merge:
I applied the changes in this PR + the suggested fixes above in a single commit here: e5e3537 I hope it's okay from your end as well! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
Two unrelated real-world PDFs failed with
Malformed FlateDecode stream.Probed at the failure point, the underlying cause is the same in both:0x0d)0x0a)A single end-of-line byte. PDF 32000-1 7.3.8.1 puts an EOL marker after the stream data and states it shall not be included in the stream length; these producers counted it.
DecompressionStreamrejects the stream outright, and the fully decoded payload is thrown away with it.The shape of the fix
The marker is accepted only after re-decoding the stream without it produces exactly the same output length. That preserves what the strict decoder was protecting: a truncated stream cannot be rescued this way, because removing a trailing byte never completes one. The re-decode only runs on the error path, so valid streams cost nothing.
Nothing else is trimmed, and this is deliberate.
testFlatealready asserts that a trailing NUL is rejected — that case still fails, untouched. So do spaces, tabs, and more than one marker. 7.3.8.1 specifies an EOL marker, not arbitrary padding, and the new cases pin that boundary in both directions:LF,CR,CRLF0x20,0x09,LF LF,LF CR,CRLF LF, and the pre-existing NULChecks
npm run test:file -- scripts/test-native-filter-semantics.mjs— passesnpm test—tsc --noEmitclean, 36 / 37 fast files passnpm run test:integration— 41 / 41npm run test:unit— 66 / 68The two failing files (
test-text-lod-core.mjs,test-room-segment-extractor.mjs) also fail onmainwithout this change; they are Windows-only path failures, unrelated.Scope
FlateDecode only. Independent of #4, #5 and #6.
Refs #2