ICE when compiling `smallvec::SmallVecData<[f64; 4]>::heap_mut`

[LDemetrios]

Hi!

So, as promised, I tried to feed your compiler a big project. And here's a panic:

thread '<unnamed>' panicked at src/lib.rs:1649:9:
failed to lower OOMIR shard canonical-types-9 to JVM bytecode: Function smallvec/SmallVecData_f64Array4::heap_mut: Failed to translate function: VerificationError { context: "Function heap_mut, block bb0, OOMIR instruction 5", message: "Failed to translate InvokeVirtual { dest: Some(\"_1_deref_object\"), class_name: \"org/rustlang/runtime/Pointer\", method_name: \"getObjectAs\", method_ty: Signature { params: [(\"self\", Pointer(Class(\"smallvec/SmallVecData_f64Array4\"))), (\"target_class\", Class(\"java/lang/String\"))], ret: Class(\"java/lang/Object\"), is_static: false }, args: [Constant(String(\"smallvec/SmallVecData_f64Array4\"))], operand: Variable { name: \"_1\", ty: Pointer(Class(\"smallvec/SmallVecData_f64Array4\")) } }: VerificationError { context: \"Function heap_mut\", message: \"could not load deferred pointer for operation getObjectAs\" }" }
stack backtrace:
   0:        0x1094ec0d8 - <std[c8fa24e64d9cf078]::backtrace::Backtrace>::create
   1:        0x1070b6bc4 - std[c8fa24e64d9cf078]::panicking::update_hook::<alloc[211306b2149fe892]::boxed::Box<rustc_driver_impl[6ce72ed364af5ef8]::install_ice_hook::{closure#1}>>::{closure#0}
   2:        0x109500418 - std[c8fa24e64d9cf078]::panicking::panic_with_hook
   3:        0x1094e07d4 - std[c8fa24e64d9cf078]::panicking::panic_handler::{closure#0}
   4:        0x1094d8d50 - std[c8fa24e64d9cf078]::sys::backtrace::__rust_end_short_backtrace::<std[c8fa24e64d9cf078]::panicking::panic_handler::{closure#0}, !>
   5:        0x1094e2040 - __rustc[897652c62be20a70]::rust_begin_unwind
   6:        0x1095e5c78 - core[f1ce509e42dc19ff]::panicking::panic_fmt
   7:        0x10c4d4c58 - rustc_codegen_jvm[d89b58a70c509517]::emit_oomir_shard::{closure#0}
   8:        0x10c3a6274 - rustc_codegen_jvm[d89b58a70c509517]::emit_oomir_shard
   9:        0x10c33a144 - std[c8fa24e64d9cf078]::sys::backtrace::__rust_begin_short_backtrace::<<rustc_codegen_jvm[d89b58a70c509517]::MyBackend as rustc_codegen_ssa[603e0dcf5e1902f0]::traits::backend::CodegenBackend>::codegen_crate::{closure#2}::{closure#0}, ()>
  10:        0x10c3c85b4 - <std[c8fa24e64d9cf078]::thread::lifecycle::spawn_unchecked<<rustc_codegen_jvm[d89b58a70c509517]::MyBackend as rustc_codegen_ssa[603e0dcf5e1902f0]::traits::backend::CodegenBackend>::codegen_crate::{closure#2}::{closure#0}, ()>::{closure#1} as core[f1ce509e42dc19ff]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
  11:        0x10950adb0 - <std[c8fa24e64d9cf078]::sys::thread::unix::Thread>::new::thread_start
  12:        0x1860d3c08 - __pthread_cond_wait


rustc version: 1.100.0-nightly (1303417c4 2026-09-21)
platform: aarch64-apple-darwinthread 'rustc' panicked at src/lib.rs:1812:30:
OOMIR shard worker stopped without a result: RecvError
stack backtrace:
   0:        0x1094ec0d8 - <std[c8fa24e64d9cf078]::backtrace::Backtrace>::create
   1:        0x1070b6bc4 - std[c8fa24e64d9cf078]::panicking::update_hook::<alloc[211306b2149fe892]::boxed::Box<rustc_driver_impl[6ce72ed364af5ef8]::install_ice_hook::{closure#1}>>::{closure#0}
   2:        0x109500418 - std[c8fa24e64d9cf078]::panicking::panic_with_hook
   3:        0x1094e07d4 - std[c8fa24e64d9cf078]::panicking::panic_handler::{closure#0}
   4:        0x1094d8d50 - std[c8fa24e64d9cf078]::sys::backtrace::__rust_end_short_backtrace::<std[c8fa24e64d9cf078]::panicking::panic_handler::{closure#0}, !>
   5:        0x1094e2040 - __rustc[897652c62be20a70]::rust_begin_unwind
   6:        0x1095e5c78 - core[f1ce509e42dc19ff]::panicking::panic_fmt
   7:        0x1095e59c4 - core[f1ce509e42dc19ff]::result::unwrap_failed
   8:        0x10c283194 - std[c8fa24e64d9cf078]::thread::scoped::scope::<<rustc_codegen_jvm[d89b58a70c509517]::MyBackend as rustc_codegen_ssa[603e0dcf5e1902f0]::traits::backend::CodegenBackend>::codegen_crate::{closure#2}, alloc[211306b2149fe892]::vec::Vec<(alloc[211306b2149fe892]::string::String, std[c8fa24e64d9cf078]::path::PathBuf)>>
   9:        0x10c3bec30 - <rustc_codegen_jvm[d89b58a70c509517]::MyBackend as rustc_codegen_ssa[603e0dcf5e1902f0]::traits::backend::CodegenBackend>::codegen_crate
  10:        0x107a1e36c - <rustc_interface[a9842193f593845c]::queries::Linker>::codegen_and_build_linker
  11:        0x107048ee0 - rustc_interface[a9842193f593845c]::passes::create_and_enter_global_ctxt::<core[f1ce509e42dc19ff]::option::Option<rustc_interface[a9842193f593845c]::queries::Linker>, rustc_driver_impl[6ce72ed364af5ef8]::run_compiler::{closure#0}::{closure#1}>
  12:        0x1070b5560 - rustc_interface[a9842193f593845c]::interface::run_compiler::<(), rustc_driver_impl[6ce72ed364af5ef8]::run_compiler::{closure#0}>::{closure#2}
  13:        0x1070a6b58 - std[c8fa24e64d9cf078]::sys::backtrace::__rust_begin_short_backtrace::<rustc_interface[a9842193f593845c]::util::run_in_thread_with_globals<rustc_interface[a9842193f593845c]::util::run_in_thread_pool_with_globals<rustc_interface[a9842193f593845c]::interface::run_compiler<(), rustc_driver_impl[6ce72ed364af5ef8]::run_compiler::{closure#0}>::{closure#2}, ()>::{closure#0}, ()>::{closure#0}::{closure#0}, ()>
  14:        0x1070ba144 - <std[c8fa24e64d9cf078]::thread::lifecycle::spawn_unchecked<rustc_interface[a9842193f593845c]::util::run_in_thread_with_globals<rustc_interface[a9842193f593845c]::util::run_in_thread_pool_with_globals<rustc_interface[a9842193f593845c]::interface::run_compiler<(), rustc_driver_impl[6ce72ed364af5ef8]::run_compiler::{closure#0}>::{closure#2}, ()>::{closure#0}, ()>::{closure#0}::{closure#0}, ()>::{closure#1} as core[f1ce509e42dc19ff]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
  15:        0x10950adb0 - <std[c8fa24e64d9cf078]::sys::thread::unix::Thread>::new::thread_start
  16:        0x1860d3c08 - __pthread_cond_wait

query stack during panic:
end of query stack

I couldn't understand what really have happened there, as to my understanding the error message isn't really helpful. Additionally, I found out that parking lot itself compiles cleanly, so it's how it used is the issue.

If you need, I tried compiling https://github.com/LDemetrios/typst-shared-library/tree/shared-lib-v2 . I probably will end up later substituting parking lot dependencies with proper JVM synchronization, but that's gonna be when I understand your compiler better xD

[LDemetrios]

It seems that some issues can be avoided by patching the code of dependencies. I hope I can get to the bottom of it some time, then I'll share what I changed so you can see what caused the issue

[LDemetrios]

It also seems that later on icu_collator it just freezes

[IntegralPilot]

Thank you for trying it out and for the report! I'm taking a look at that crash now, it just looks like a small bug that should be easy to fix. It's happening when compiling smallvec::SmallVecData<[f64; 4]>::heap_mut. I'll look at the freeze after.

[IntegralPilot]

MRE (can probably be a bit smaller still, but it's just what I'm working with right now)

use std::mem::{ManuallyDrop, MaybeUninit};
use std::ptr::NonNull;

trait Array {
    type Item;
}
impl Array for [f64; 4] {
    type Item = f64;
}

union Data<A: Array> {
    inline: ManuallyDrop<MaybeUninit<A>>,
    heap: (NonNull<A::Item>, usize),
}

impl<A: Array> Data<A> {
    unsafe fn heap_mut(&mut self) -> (NonNull<A::Item>, &mut usize) {
        let heap = unsafe { &mut self.heap };
        (heap.0, &mut heap.1)
    }

    unsafe fn inline_mut(&mut self) -> NonNull<A::Item> {
        unsafe { NonNull::new_unchecked(self.inline.as_mut_ptr() as *mut A::Item) }
    }
}

pub fn run() {
    let mut data = Data::<[f64; 4]> {
        inline: ManuallyDrop::new(MaybeUninit::new([1.0, 2.0, 3.0, 4.0])),
    };
    unsafe {
        let pointer = data.inline_mut();
        assert_eq!(*pointer.as_ptr().add(2), 3.0);
    }
    let mut heap = [5.0, 6.0, 7.0, 8.0, 9.0];
    data = Data {
        heap: (NonNull::new(heap.as_mut_ptr()).unwrap(), heap.len()),
    };
    unsafe {
        let (pointer, len) = data.heap_mut();
        assert_eq!(pointer.as_ptr(), heap.as_mut_ptr());
        *len -= 1;
        pointer.as_ptr().add(4).write(99.0);
        assert_eq!(data.heap.1, 4);
        assert_eq!(heap[4], 99.0);
    }
}

[LDemetrios]

Thank you for trying it out and for the report! I'm taking a look at that crash now, it just looks like a small bug that should be easy to fix. It's happening when compiling smallvec::SmallVecData<[f64; 4]>::heap_mut. I'll look at the freeze after.

I actually think I found the freeze. It's in oomir::Function, specifically constant tables, which results in 34GB footprint for icu_* crates before the compiler crashes with OOM on my machine

[IntegralPilot]

I actually think I found the freeze. It's in oomir::Function, specifically constant tables, which results in 34GB footprint for icu_* crates before the compiler crashes with OOM on my machine

Oh perfect, thank you! Can you try it with the experimental branch? I cleaned up a bunch of memory usage there. It's here: #57

[IntegralPilot]

But I will still look into that memory issue and make a fix for mainline, because experimental isn't ready to be released yet.

[IntegralPilot]

The crash issue was a tiny fix, I made a very silly mistake sorry! Pushed here, will wait for CI to go green before calling it fixed. Can you also try pulling the change and confirming it fixes it on your end too?

2435278

[IntegralPilot]

OK CI's passed and I'm going to bed now but I'll look into how to fix the freeze on main tomorrow.

[LDemetrios]

Oh, good night!
I've been fixing some issues with my local rust setup and only now was able to start a new compilation with memory improvements, still waiting. Thanks for your fast responses!
Well, I've encountered several issues after working around this one, not sure if they are all related or different, I'll be testing that later as well. For now not doing anything crazy, just trying to compile xD.