fix(parquet/variant): validate metadata offset tables - #1063
Open
fallintoplace wants to merge 2 commits into
Open
fix(parquet/variant): validate metadata offset tables#1063fallintoplace wants to merge 2 commits into
fallintoplace wants to merge 2 commits into
Conversation
serramatutu
reviewed
Jul 30, 2026
serramatutu
left a comment
Contributor
There was a problem hiding this comment.
Ran the test locally before and after the fix and it all passes. LGTM to me except for a the small mistake >= instead of > when checking the range.
| return fmt.Errorf("%w: offset out of range: %d > %d", | ||
| ErrInvalidMetadata, (dictSize+hdrSizeBytes)*uint32(offsetSz), len(m.data)) | ||
| valuesStart := uint64(hdrSizeBytes) + (uint64(dictSize)+2)*uint64(offsetSz) | ||
| if valuesStart > uint64(len(m.data)) { |
Contributor
Author
There was a problem hiding this comment.
valuesStart == len(m.data) is valid when the offset table ends exactly at the input boundary and the dictionary’s strings contain zero total bytes.
Changing > to >= would incorrectly reject that valid encoding. The subsequent check already rejects any nonzero final offset because valuesStart + end would exceed the input length.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Validate the complete Variant metadata offset table before allocating the key slice or reading offsets. The parser now also rejects a nonzero initial offset, decreasing offsets, and string offsets beyond the input.
Why
The previous size check omitted the final offset entry. Truncated input could therefore reach an out-of-bounds slice, while decreasing offsets could underflow during key-size calculation. A large untrusted dictionary count was also used to allocate before checking whether the offset table could fit in the input.
Malformed metadata now consistently returns an error wrapping
ErrInvalidMetadata.Testing
go test ./parquet/variant