Skip to content

ORC, Flink: Avoid redundant java.time allocations when reading ORC timestamps - #16731

Open
wombatu-kun wants to merge 1 commit into
apache:mainfrom
wombatu-kun:orc-timestamp-reader-alloc
Open

ORC, Flink: Avoid redundant java.time allocations when reading ORC timestamps#16731
wombatu-kun wants to merge 1 commit into
apache:mainfrom
wombatu-kun:orc-timestamp-reader-alloc

Conversation

@wombatu-kun

@wombatu-kun wombatu-kun commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

The generic and Flink ORC timestamp value readers build throwaway java.time objects on every value. Reading a timestamp/timestamptz column went through Instant.ofEpochSecond(...).atOffset(ZoneOffset.UTC)[...], which allocates, per value: a transient Instant; an OffsetDateTime that is immediately discarded (for the readers that ultimately return a LocalDateTime/TimestampData); and, less obviously, a ZoneRules object plus a small backing array, because instant.atOffset(ZoneOffset.UTC) routes through OffsetDateTime.ofInstant(instant, zone), which calls zone.getRules(), and ZoneOffset.getRules() returns a freshly allocated ZoneRules on every call.

This replaces those expressions with direct constructions that pass the ZoneOffset straight to the factory methods and never call getRules(). For example, in GenericOrcReaders:

// TimestampReader (-> LocalDateTime)
-  Instant.ofEpochSecond(Math.floorDiv(tcv.time[row], 1_000), tcv.nanos[row])
-      .atOffset(ZoneOffset.UTC)
-      .toLocalDateTime();
+  LocalDateTime.ofEpochSecond(Math.floorDiv(tcv.time[row], 1_000), tcv.nanos[row], ZoneOffset.UTC);

The timestamptz readers follow the same shape (OffsetDateTime.of(LocalDateTime.ofEpochSecond(...), UTC) in the generic reader; TimestampData.fromInstant(Instant.ofEpochSecond(...)) in Flink). The Flink TimestampTzReader additionally did Instant.ofEpochSecond(...).atOffset(UTC).toInstant(), an identity round-trip back to an Instant, which is now removed.

All rewrites are behavior-preserving. instant.atOffset(UTC).toLocalDateTime() is exactly what LocalDateTime.ofEpochSecond(seconds, nanoOfSecond, UTC) produces, and the readers always pass tcv.nanos[row], which is in [0, 1_000_000_000) (it carries java.sql.Timestamp.getNanos() semantics; the whole second and the sign live in tcv.time and are split out with Math.floorDiv), so the nanoOfSecond range contract of LocalDateTime.ofEpochSecond is always satisfied.

Affected readers:

  • iceberg-orc GenericOrcReaders (TimestampReader, TimestampTzReader) - the generic Record ORC path, used by IcebergGenerics and by delete-file reads via BaseDeleteLoader.
  • iceberg-flink FlinkOrcReaders for v2.1 (TimestampReader, TimestampTzReader) - the Flink ORC scan path. The change is identical for v1.20 and v2.0; a backport PR will follow.

Benchmarks (JMH, JDK 17, SingleShotTime, gc profiler; allocation is gc.alloc.rate.norm, which is deterministic):

Generic Record read, 2.5M rows x 8 timestamp columns (20M values/op):

Metric Before After
time 2.431 s/op 1.983 s/op (-18%)
allocation 7.434 GB/op 5.034 GB/op (-32%, -120 B/value)

Flink TimestampData readers, 10.24M values/op:

Reader time (before -> after) allocation (before -> after)
timestamp 0.358 -> 0.291 s/op 144 -> 24 B/value (-83%)
timestamptz 0.380 -> 0.070 s/op 168 -> 24 B/value (-86%)

After the change both Flink readers bottom out at the 24-byte TimestampData result, because simplifying the expression also lets the JIT scalar-replace the residual LocalDateTime/LocalDate/LocalTime (in the original, the atOffset detour and the escaping ZoneRules defeated escape analysis).

Testing: covered by existing round-trip tests - TestGenericData (parameterized over timestamp, timestamptz, timestamp_ns, timestamptz_ns, including negative epochs and a cross-timezone write/read) and TestFlinkOrcReaderWriter; both pass with the change.


AI Disclosure

  • Model: Claude Opus 4.8
  • Platform/Tool: Claude Code
  • Human Oversight: fully reviewed
  • Prompt Summary: Remove the redundant per-value Instant and OffsetDateTime allocations from the generic and Flink ORC timestamp readers.

@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Jul 10, 2026
@wombatu-kun

Copy link
Copy Markdown
Contributor Author

no stale

@github-actions github-actions Bot removed the stale label Jul 11, 2026
…mestamps

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@wombatu-kun
wombatu-kun force-pushed the orc-timestamp-reader-alloc branch from 8d1b21b to c18482a Compare August 4, 2026 02:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant