diff --git a/_posts/2026-09-29-arrow-rs-60.0.0.md b/_posts/2026-09-29-arrow-rs-60.0.0.md new file mode 100644 index 00000000000..8e8e7271756 --- /dev/null +++ b/_posts/2026-09-29-arrow-rs-60.0.0.md @@ -0,0 +1,389 @@ +--- +layout: post +title: "Apache Arrow Rust 60.0.0 Release" +date: "2026-09-29 00:00:00" +author: Richard Baah & pmc +categories: [release] +--- + + +The Apache Arrow team is pleased to announce that the v60.0.0 release of Apache Arrow +Rust is now available on crates.io ([arrow] and [parquet]) and as [source download]. + +[arrow]: https://crates.io/crates/arrow +[parquet]: https://crates.io/crates/parquet +[source download]: https://dist.apache.org/repos/dist/release/arrow/arrow-rs-60.0.0 + +See the [60.0.0 changelog] for a full list of changes. + +[60.0.0 changelog]: https://github.com/apache/arrow-rs/blob/60.0.0/CHANGELOG.md + +## Highlights + +This release includes new Parquet encoding support, a sustained effort to remove +panics from the codebase, a large number of kernel performance improvements, and +many bug fixes across decimals, take/filter, and FFI. + +MSRV is bumped to 1.88, +`rand` upgraded to 0.10, `object_store` to 0.14.1, and several long-deprecated +items have been removed. See the [60.0.0 changelog] for the full breaking +changes list. + +## New Features + +### Parquet ALP Encoder/Decoder Support + +[@sdf-jkl] added support for the [Adaptive Lossless Floating-Point (ALP)] +encoding in Parquet ([#9372]). ALP is a high-performance encoding for +floating-point columns that enables compact storage, fast decoding and single value random-access. +This makes arrow-rs the first Parquet implementations to support ALP, +keeping the [parquet] crate aligned with the latest Parquet specification. + +[Adaptive Lossless Floating-Point (ALP)]: https://parquet.apache.org/blog/2026/09/22/alp-adaptive-lossless-floating-point-encoding-in-apache-parquet/ +[@sdf-jkl]: https://github.com/sdf-jkl +[#9372]: https://github.com/apache/arrow-rs/pull/9372 + +### Parquet Page Index Improvements + +[@etseidl] made several improvements to Parquet page index handling: a new +[`PageIndex`] struct encapsulating column and offset indexes ([#10719]), new +[`PageIndexBuilder`] and [`PageIndexProvider`] types ([#10842]), and page index +decoders are now public ([#10899]). One major use case enabled by these +changes is supplying external or cached `PageIndex`es directly to the reader +(for example, indexes previously fetched and stored outside the file), +avoiding redundant I/O on repeated scans. This also opens the door for +selective population of the page indexes which will help reduce the latency +of filtered scans against very wide tables. + +[@etseidl]: https://github.com/etseidl +[`PageIndex`]: https://docs.rs/parquet/60.0.0/parquet/file/metadata/page_index/struct.PageIndex.html +[`PageIndexBuilder`]: https://docs.rs/parquet/60.0.0/parquet/file/metadata/page_index/struct.PageIndexBuilder.html +[`PageIndexProvider`]: https://docs.rs/parquet/60.0.0/parquet/file/metadata/page_index/trait.PageIndexProvider.html +[#10719]: https://github.com/apache/arrow-rs/pull/10719 +[#10842]: https://github.com/apache/arrow-rs/pull/10842 +[#10899]: https://github.com/apache/arrow-rs/pull/10899 + +### Parquet NDV Statistics, FILE Logical Type, and IEEE 754 Total Order + +[@Rich-T-kid] implemented writing num distinct values (NDV) statistics in the +Parquet writer ([#10654]) and added row-group-level distinct counts to +`StatisticsConverter` ([#10652]). Query engines use these to make better +planning decisions such as join strategy selection and filter selectivity +estimates. + +[@brkyvz] added support for the `FILE` logical type ([#10109]). [@etseidl] +implemented [PARQUET-2249] IEEE 754 total order for floating-point comparisons +([#9619]) and the corresponding `INT96` timestamp `ColumnOrder` ([#10106]). +[@etseidl] also removed the previous limit of 32,768 row groups per file +([#10149]). + +[@Rich-T-kid]: https://github.com/Rich-T-kid +[@brkyvz]: https://github.com/brkyvz +[PARQUET-2249]: https://issues.apache.org/jira/browse/PARQUET-2249 +[#10654]: https://github.com/apache/arrow-rs/pull/10654 +[#10652]: https://github.com/apache/arrow-rs/pull/10652 +[#10109]: https://github.com/apache/arrow-rs/pull/10109 +[#9619]: https://github.com/apache/arrow-rs/pull/9619 +[#10106]: https://github.com/apache/arrow-rs/pull/10106 +[#10149]: https://github.com/apache/arrow-rs/pull/10149 + +### New `arrow-cmp` Crate + +[@Rich-T-kid] introduced the [`arrow-cmp`] crate ([#10325]), splitting +comparison kernel logic out of [`arrow-ord`] into its own focused crate. This +gives users a more targeted dependency for Arrow comparison operations without +pulling in the broader `arrow-ord` crate. + +[`arrow-ord`]: https://crates.io/crates/arrow-ord + +[`arrow-cmp`]: https://crates.io/crates/arrow-cmp +[#10325]: https://github.com/apache/arrow-rs/pull/10325 + +## Panic Elimination + +A lot of work went into converting panics into proper `Result` returns this +release. [@emilk] fixed `try_` functions still panicking in edge cases ([#10730], +[#10755]) and removed panics from `GenericByteArray::from_iter_values` ([#10729]). +[@bit2swaz] replaced panics with errors in IPC schema and footer parsing +([#10647], [#10744]) and closed an FFI `Drop` soundness hole ([#10431]). +[@Rich-T-kid] fixed a potential use-after-free in `Buffer::shrink_to_fit` ([#10932]) +and converted `MutableBuffer` panicking callsites to fallible methods ([#10641], +[#10317]). [@okhsunrog] made `interleave`/`concat` return errors on dictionary key +overflow ([#10675]). [@ranflarion] converted a Parquet buffer push panic into an +error ([#10564]) and [@dhruvxvaishnav] did the same for invalid dictionary index +bit widths ([#10725]). + +[@emilk]: https://github.com/emilk +[@bit2swaz]: https://github.com/bit2swaz +[@okhsunrog]: https://github.com/okhsunrog +[@ranflarion]: https://github.com/ranflarion +[@dhruvxvaishnav]: https://github.com/dhruvxvaishnav +[#10730]: https://github.com/apache/arrow-rs/pull/10730 +[#10755]: https://github.com/apache/arrow-rs/pull/10755 +[#10729]: https://github.com/apache/arrow-rs/pull/10729 +[#10647]: https://github.com/apache/arrow-rs/pull/10647 +[#10744]: https://github.com/apache/arrow-rs/pull/10744 +[#10431]: https://github.com/apache/arrow-rs/pull/10431 +[#10932]: https://github.com/apache/arrow-rs/pull/10932 +[#10641]: https://github.com/apache/arrow-rs/pull/10641 +[#10317]: https://github.com/apache/arrow-rs/pull/10317 +[#10675]: https://github.com/apache/arrow-rs/pull/10675 +[#10564]: https://github.com/apache/arrow-rs/pull/10564 +[#10725]: https://github.com/apache/arrow-rs/pull/10725 + +## Performance Improvements + +This release has a large number of performance improvements across kernels, +casting, and Parquet I/O. + +### Take/Filter Kernel Improvements + +[@Rich-T-kid] landed a series of take/filter kernel speedups this release: +`take` on `FixedSizeList` is up to **3.3x faster** ([#10441]), `take` on +`List` up to **1.8x faster** ([#10812]), `take` on boolean and null buffers +up to **1.7x faster** ([#10813]), and `filter` on `FixedSizeBinary` up to +**1.4x faster** ([#10993]). A `NullBuffer::expand` rewrite for aligned counts +delivers up to **143x faster** buffer expansion ([#10976], [#10980]). + +[@YUZHEthefool] improved `rank` on `StringViewArray` by up to **3.2x faster** +via prefix key caching ([#10605]). [@giladkl] made equality/inequality +comparisons of byte-view arrays against short scalars up to **2.5x faster** +([#10689]). [@cakeni] replaced `BufferBuilder` with `Vec` across +`concat_elements`, `take_run`, `substring`, and other kernels +([#10629]-[#10634]), and [@kowanietz] did the same for primitive unary +operations ([#10783]) and fixed-size binary take ([#10773]), cutting +unnecessary allocations across the board. [@haohuaijin] fixed in-place +bitwise ops corrupting bits outside the requested range ([#10444]). + +### Decimal Casting + +[@neilconway] made decimal parsing dramatically faster across the board. A +foundational rewrite of the decimal-from-string path ([#10668]) alone delivers +**up to ~8x faster** casting from strings to `Decimal128` and up to **92% lower parsing time** +on individual parse microbenchmarks. A follow-up pass for separate +integer/fractional digit scanning ([#10974]) added another **5–15%** on top, +and skipping redundant precision checks ([#10998]) added up to **~10%** more. +Formatting `i256` without `num-bigint` ([#11000]) rounds things out with +**~1.8x faster** `Decimal256` formatting for typical 38-digit values. + +### Parquet Reading Improvements + +Plain Parquet string column reads into dictionary arrays are **~1.5x faster** +([#10614]). [@adriangb] sped up `DELTA_BYTE_ARRAY` shared-prefix decoding by +scanning a block at a time ([#10549]), reaching **up to 19x faster** on the +prefix-scan microbenchmark and **4.2x faster** end-to-end for small +shared-prefix strings. [@haohuaijin] removed redundant copies in Parquet +mask-backed intersection/union ([#10446]), delivering **2–6x faster** mask +operations depending on alignment. [@hhhizzz] fixed cached Mask reads crossing +unloaded sparse pages ([#10735]). + +### Binary Size Reduction + +[@alamb] reduced binary size by about **2%** by monomorphizing `PrimitiveArray` +to `ArrayData` conversion and `Debug` formatting helpers ([#10893], [#10890]) — +the Debug change alone cut **~12% of LLVM IR** from the `cast_kernels` binary. + +[@neilconway]: https://github.com/neilconway +[@haohuaijin]: https://github.com/haohuaijin +[@hhhizzz]: https://github.com/hhhizzz +[@alamb]: https://github.com/alamb +[@adriangb]: https://github.com/adriangb +[@YUZHEthefool]: https://github.com/YUZHEthefool +[@giladkl]: https://github.com/giladkl +[@cakeni]: https://github.com/cakeni +[@kowanietz]: https://github.com/kowanietz +[#10812]: https://github.com/apache/arrow-rs/pull/10812 +[#10441]: https://github.com/apache/arrow-rs/pull/10441 +[#10813]: https://github.com/apache/arrow-rs/pull/10813 +[#10993]: https://github.com/apache/arrow-rs/pull/10993 +[#10976]: https://github.com/apache/arrow-rs/pull/10976 +[#10980]: https://github.com/apache/arrow-rs/pull/10980 +[#10614]: https://github.com/apache/arrow-rs/pull/10614 +[#10974]: https://github.com/apache/arrow-rs/pull/10974 +[#10998]: https://github.com/apache/arrow-rs/pull/10998 +[#10668]: https://github.com/apache/arrow-rs/pull/10668 +[#11000]: https://github.com/apache/arrow-rs/pull/11000 +[#10893]: https://github.com/apache/arrow-rs/pull/10893 +[#10890]: https://github.com/apache/arrow-rs/pull/10890 +[#10549]: https://github.com/apache/arrow-rs/pull/10549 +[#10446]: https://github.com/apache/arrow-rs/pull/10446 +[#10605]: https://github.com/apache/arrow-rs/pull/10605 +[#10689]: https://github.com/apache/arrow-rs/pull/10689 +[#10629]: https://github.com/apache/arrow-rs/pull/10629 +[#10634]: https://github.com/apache/arrow-rs/pull/10634 +[#10783]: https://github.com/apache/arrow-rs/pull/10783 +[#10773]: https://github.com/apache/arrow-rs/pull/10773 + +## Bug Fixes + +[@neilconway] fixed a cluster of decimal bugs including incorrect `Decimal256` +casting to signed integers ([#10857]), wrong min/max statistics for `BYTE_ARRAY` +decimals ([#10861]), mismatched scales in `make_comparator` ([#10864]), +formatting bugs ([#10869]), and a spurious assert when `scale == precision` +([#10875]). + +[@yongster] fixed `take` for null indices in dense union and run-end encoded +arrays ([#10909]). + +[@1fanwang] and [@tpoterba] fixed `take` and `interleave` on zero-width +`FixedSizeListArray` ([#10915], [#11026]). + +[@jaideeppyne] fixed struct `ArrayData` slicing double-counting the offset +([#10835], [#10934]). [@linhongyu510] fixed struct null validation not +accounting for parent offsets ([#10970]). + +[@PlenoraETL] made the IPC reader return an error for a `DictionaryBatch` +missing its data instead of silently producing wrong results ([#11020]). +[@bit2swaz] made `FFI_ArrowSchema::with_metadata` `unsafe` to close a +soundness gap ([#10764]). + +[@adriangb] fixed `DELTA_BYTE_ARRAY` deduplication for values larger than +the page size limit ([#10505]). + +[@yongster]: https://github.com/yongster +[@1fanwang]: https://github.com/1fanwang +[@tpoterba]: https://github.com/tpoterba +[@jaideeppyne]: https://github.com/jaideeppyne +[@linhongyu510]: https://github.com/linhongyu510 +[@PlenoraETL]: https://github.com/PlenoraETL +[#10444]: https://github.com/apache/arrow-rs/pull/10444 +[#10735]: https://github.com/apache/arrow-rs/pull/10735 + +[#10857]: https://github.com/apache/arrow-rs/pull/10857 +[#10861]: https://github.com/apache/arrow-rs/pull/10861 +[#10864]: https://github.com/apache/arrow-rs/pull/10864 +[#10869]: https://github.com/apache/arrow-rs/pull/10869 +[#10875]: https://github.com/apache/arrow-rs/pull/10875 +[#10909]: https://github.com/apache/arrow-rs/pull/10909 +[#10915]: https://github.com/apache/arrow-rs/pull/10915 +[#11026]: https://github.com/apache/arrow-rs/pull/11026 +[#10835]: https://github.com/apache/arrow-rs/pull/10835 +[#10934]: https://github.com/apache/arrow-rs/pull/10934 +[#10970]: https://github.com/apache/arrow-rs/pull/10970 +[#11020]: https://github.com/apache/arrow-rs/pull/11020 +[#10764]: https://github.com/apache/arrow-rs/pull/10764 +[#10505]: https://github.com/apache/arrow-rs/pull/10505 + +## Other Notable Changes + +- [@adamreeve] added round-trip support for `Dictionary(_, Utf8View/BinaryView)` + columns in Parquet ([#10831]) +- [@ranflarion] improved Parquet bloom filters: populate from the dictionary + while a column is dictionary encoded ([#10966]) and added a writer option to + skip bloom filters for all-dictionary column chunks ([#10963]) +- [@haohuaijin] added row-group-local `RowSelection` support to the push + decoder ([#10702]) +- [@1fanwang] added `Utf8View` and `BinaryView` support to `substring` ([#10672]) +- [@etseidl] removed previously deprecated Parquet functions ([#10565]) and + deprecated `ColumnOrder::sort_order_for_type` ([#10104]) as a follow-up to + the INT96/IEEE 754 correctness work — both are breaking changes users will + notice at compile time. + +[@adamreeve]: https://github.com/adamreeve +[#10831]: https://github.com/apache/arrow-rs/pull/10831 +[#10966]: https://github.com/apache/arrow-rs/pull/10966 +[#10963]: https://github.com/apache/arrow-rs/pull/10963 +[#10702]: https://github.com/apache/arrow-rs/pull/10702 +[#10672]: https://github.com/apache/arrow-rs/pull/10672 +[#10565]: https://github.com/apache/arrow-rs/pull/10565 +[#10104]: https://github.com/apache/arrow-rs/pull/10104 + +## Thanks to Our Contributors + +```console +$ git shortlog -sn 59.2.0..60.0.0 + 38 Richard Baah + 22 Jeffrey Vo + 19 Andrew Lamb + 17 Emil Ernerfeldt + 16 Neil Conway + 12 Adrian Garcia Badaracco + 11 WaterWhisperer + 11 cakeni + 10 Ed Seidl + 8 Kosta Tarasov + 6 Huaijin + 6 ranflarion + 5 Aditya Mishra + 5 yongster + 4 Stefan Wang + 3 Abhishek + 3 Dhruv Vaishnav + 3 Thefool + 3 WissssleyL + 2 Andrea Bozzo + 2 Bharadwaj Pendyala + 2 Emily Matheys + 2 Ethan Tang + 2 Giladi + 2 Jaideep Pyne + 2 Oleks V + 2 Peter L + 2 Peter Lee + 2 Xinyao Zhang + 2 Yiming Qiao + 2 hylin + 2 pawan + 1 AarryaSaraf + 1 Adam Reeve + 1 Adam Reichold + 1 Alexander Rafferty + 1 Ali Asghar + 1 Ben Kowanietz + 1 Burak Yavuz + 1 Chao Sun + 1 Congxian Qiu + 1 Connor Tsui + 1 Danila Gornushko + 1 Fredrik Fornwall + 1 Han You + 1 Haresh Khanna + 1 Harshal Joshi + 1 Huang Qiwei + 1 Johan Vaz + 1 Jonas Dedden + 1 Jordan Epstein + 1 Langning Zhang + 1 Liam Bao + 1 Marcelo Tesla + 1 Marco Bonamente + 1 Michal Piatkowski + 1 MsfPablo + 1 Narendran K T + 1 Nuno Faria + 1 Phoenix + 1 Qi Zhu + 1 Raghvendra Singh + 1 Raz Luvaton + 1 Seowoo Jang + 1 Thor + 1 Tim Poterba + 1 Yin Li + 1 anchor + 1 kowanietz + 1 linfeng + 1 subotac + 1 wterrr + 1 yanglongwei + 1 张跃哲 +``` + +Thanks to everyone who contributed bug reports, reviews, and pull requests for +this release!