Skip to content

Flink: Avoid redundant java.time allocations in ORC timestamp writers - #16733

Open
wombatu-kun wants to merge 1 commit into
apache:mainfrom
wombatu-kun:flink-orc-timestamp-writer-alloc
Open

Flink: Avoid redundant java.time allocations in ORC timestamp writers#16733
wombatu-kun wants to merge 1 commit into
apache:mainfrom
wombatu-kun:flink-orc-timestamp-writer-alloc

Conversation

@wombatu-kun

@wombatu-kun wombatu-kun commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #16731, which removed the same redundant java.time allocations from the ORC timestamp readers (generic and Flink). This PR does the writer side for Flink.

FlinkOrcWriters defines four ORC timestamp writer classes - TimestampWriter, TimestampTzWriter, TimestampNanoWriter and TimestampNanoTzWriter - each of which converted a TimestampData to a java.time object per value to derive the ORC column-vector fields. The no-zone writers (TimestampWriter, TimestampNanoWriter) used data.toInstant().atOffset(ZoneOffset.UTC), which allocates, per value, a transient Instant, an OffsetDateTime, and a ZoneRules (plus a small backing array) - because instant.atOffset(ZoneOffset.UTC) routes through OffsetDateTime.ofInstant(instant, zone), which calls zone.getRules(), and ZoneOffset.getRules() returns a freshly allocated ZoneRules on every call.

TimestampData already exposes the values ORC needs without any allocation: getMillisecond() (epoch millis) and getNanoOfMillisecond(). This rewrites all four writers to read those directly and fill the TimestampColumnVector with plain arithmetic. ORC's TimestampColumnVector stores time as epoch millis and nanos as nanoseconds-within-the-second, so:

long millis = data.getMillisecond();
cv.time[rowId] = millis;
int nanosOfSecond = (int) Math.floorMod(millis, 1_000L) * 1_000_000 + data.getNanoOfMillisecond();
cv.nanos[rowId] = nanosOfSecond / 1_000 * 1_000;   // micros writers; the nano writers keep nanosOfSecond

This is behavior-preserving. The old code computed cv.time as instant.toEpochMilli(), which equals data.getMillisecond(), and cv.nanos from the instant's nanos-of-second, which equals Math.floorMod(millis, 1000) * 1_000_000 + data.getNanoOfMillisecond(); Math.floorMod yields the correct non-negative milli-of-second for pre-1970 (negative) timestamps. The now-unused java.time imports and two stale @SuppressWarnings("JavaInstantGetSecondsGetNano") are removed. The change touches all four writer classes in FlinkOrcWriters. This PR applies it to v2.1 only; it is identical for v1.20 and v2.0, which a backport PR will follow up on.

Benchmark (JMH, JDK 17, SingleShotTime, gc profiler; allocation is gc.alloc.rate.norm, which is deterministic), driving the value writers over a reused TimestampColumnVector, 10.24M writes/op:

Writer time (before -> after) allocation (before -> after)
timestamp (no zone) 0.376 -> 0.026 s/op 144 -> ~0 B/value
timestamptz 0.073 -> 0.030 s/op already ~0 B/value

The no-zone writers carried the real allocation (the ZoneRules from atOffset); the tz writers were already allocation-free (the JIT scalar-replaces their Instant) but still gain from dropping the Instant/toEpochMilli work. The generic data-module ORC writers were checked too, but they use toInstant().toEpochMilli() without atOffset, which the JIT already scalar-replaces even on the end-to-end write path (no measurable allocation), so they are intentionally left unchanged.

Testing: covered by the existing TestFlinkOrcReaderWriter round-trip test (writes then reads back, including pre-1970 timestamps and multiple timezones); passes for Flink 2.1.


AI Disclosure

@github-actions github-actions Bot added the flink label Jun 9, 2026
@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Jul 10, 2026
@wombatu-kun

Copy link
Copy Markdown
Contributor Author

no stale

@github-actions github-actions Bot removed the stale label Jul 11, 2026
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@wombatu-kun
wombatu-kun force-pushed the flink-orc-timestamp-writer-alloc branch from ece1d63 to c6a7bd9 Compare August 4, 2026 02:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant