Skip to content

GH-3679: Allow TimestampType to annotate FLBA(12)#3680

Open
divjotarora wants to merge 1 commit into
apache:masterfrom
divjotarora:flba12-ts
Open

GH-3679: Allow TimestampType to annotate FLBA(12)#3680
divjotarora wants to merge 1 commit into
apache:masterfrom
divjotarora:flba12-ts

Conversation

@divjotarora

Copy link
Copy Markdown
Contributor

Rationale for this change

See parquet-format issue apache/parquet-format#600 and the linked proposal document for rationale

What changes are included in this PR?

This PR implements support for the TimestampType logical type annotation on FIXED_LEN_BYTE_ARRAY physical type values with type_length=12 (96 bits).

Are these changes tested?

Yes, several test files are updated along with the source.

Are there any user-facing changes?

Users will be able to declare schemas with this physical/logical type combination. No API changes.

Closes #3679

@stevomitric stevomitric left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the converted_type be suppressed for the FLBA(12) carrier, the same way NANOS already returns empty? Those legacy types are defined as INT64-only, so emitting them here gives a contradictory footer.

if (primitiveType == PrimitiveTypeName.FIXED_LEN_BYTE_ARRAY) {
return checkFixedPrimitiveType(12, timestampLogicalType);
}
return checkInt64PrimitiveType(timestampLogicalType);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this throw an error saying timestamps only support int64 physical type - i.e. when we pass in int32? Should we extend the error message to include FLBA as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support extended precision for nanosecond timestamps

2 participants