Flink: Avoid redundant java.time allocations in ORC timestamp writers - #16733
Open
wombatu-kun wants to merge 1 commit into
Open
Flink: Avoid redundant java.time allocations in ORC timestamp writers#16733wombatu-kun wants to merge 1 commit into
wombatu-kun wants to merge 1 commit into
Conversation
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
Contributor
Author
|
no stale |
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
wombatu-kun
force-pushed
the
flink-orc-timestamp-writer-alloc
branch
from
August 4, 2026 02:41
ece1d63 to
c6a7bd9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #16731, which removed the same redundant
java.timeallocations from the ORC timestamp readers (generic and Flink). This PR does the writer side for Flink.FlinkOrcWritersdefines four ORC timestamp writer classes -TimestampWriter,TimestampTzWriter,TimestampNanoWriterandTimestampNanoTzWriter- each of which converted aTimestampDatato ajava.timeobject per value to derive the ORC column-vector fields. The no-zone writers (TimestampWriter,TimestampNanoWriter) useddata.toInstant().atOffset(ZoneOffset.UTC), which allocates, per value, a transientInstant, anOffsetDateTime, and aZoneRules(plus a small backing array) - becauseinstant.atOffset(ZoneOffset.UTC)routes throughOffsetDateTime.ofInstant(instant, zone), which callszone.getRules(), andZoneOffset.getRules()returns a freshly allocatedZoneRuleson every call.TimestampDataalready exposes the values ORC needs without any allocation:getMillisecond()(epoch millis) andgetNanoOfMillisecond(). This rewrites all four writers to read those directly and fill theTimestampColumnVectorwith plain arithmetic. ORC'sTimestampColumnVectorstorestimeas epoch millis andnanosas nanoseconds-within-the-second, so:This is behavior-preserving. The old code computed
cv.timeasinstant.toEpochMilli(), which equalsdata.getMillisecond(), andcv.nanosfrom the instant's nanos-of-second, which equalsMath.floorMod(millis, 1000) * 1_000_000 + data.getNanoOfMillisecond();Math.floorModyields the correct non-negative milli-of-second for pre-1970 (negative) timestamps. The now-unusedjava.timeimports and two stale@SuppressWarnings("JavaInstantGetSecondsGetNano")are removed. The change touches all four writer classes inFlinkOrcWriters. This PR applies it to v2.1 only; it is identical for v1.20 and v2.0, which a backport PR will follow up on.Benchmark (JMH, JDK 17,
SingleShotTime,gcprofiler; allocation isgc.alloc.rate.norm, which is deterministic), driving the value writers over a reusedTimestampColumnVector, 10.24M writes/op:The no-zone writers carried the real allocation (the
ZoneRulesfromatOffset); the tz writers were already allocation-free (the JIT scalar-replaces theirInstant) but still gain from dropping theInstant/toEpochMilliwork. The generic data-module ORC writers were checked too, but they usetoInstant().toEpochMilli()withoutatOffset, which the JIT already scalar-replaces even on the end-to-end write path (no measurable allocation), so they are intentionally left unchanged.Testing: covered by the existing
TestFlinkOrcReaderWriterround-trip test (writes then reads back, including pre-1970 timestamps and multiple timezones); passes for Flink 2.1.AI Disclosure