avr: SIGSEGV (invalid memory reference) when compiling core

[tones111]

The AVR Rust book describes the need to build the core library when compiling a binary. When attempting to compile a simple binary the following error occurs:

$ cargo build -Z build-std=core --target avr-unknown-gnu-atmega328 --release
...
error: could not compile `core`

Caused by:
process didn't exit successfully: rustc --crate-name core --edition=2021 /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=568ee322bbc8e175 -C extra-filename=-568ee322bbc8e175 --out-dir /home/paul/src/hello/target/avr-unknown-gnu-atmega328/release/deps --target avr-unknown-gnu-atmega328 -Z force-unstable-if-unmarked -L dependency=/home/paul/src/hello/target/avr-unknown-gnu-atmega328/release/deps -L dependency=/home/paul/src/hello/target/release/deps --cap-lints allow (signal: 11, SIGSEGV: invalid memory reference)
`

Trying with different nightly toolchains this appears to have been introduced between...
works: rustc 1.66.0-nightly (4a14677 2022-09-23)
fails: rustc 1.66.0-nightly (3f83906 2022-09-24)

The following is a backtrace obtained from running gdb on the core dump:

#0 0x00007f4ed78fa7e8 in llvm::MachineInstr::addRegisterDead(llvm::Register, llvm::TargetRegisterInfo const*, bool) ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.66.0-nightly.so
#1 0x00007f4ed8258236 in llvm::LiveIntervals::computeDeadValues(llvm::LiveInterval&, llvm::SmallVectorImplllvm::MachineInstr*) ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.66.0-nightly.so
#2 0x00007f4ed8257645 in llvm::LiveIntervals::shrinkToUses(llvm::LiveInterval, llvm::SmallVectorImplllvm::MachineInstr**) ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.66.0-nightly.so
#3 0x00007f4ed8256ea7 in llvm::LiveRangeEdit::eliminateDeadDefs(llvm::SmallVectorImplllvm::MachineInstr*&, llvm::ArrayRefllvm::Register) ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.66.0-nightly.so
#4 0x00007f4ed8635b0d in (anonymous namespace)::InlineSpiller::postOptimization() [clone .llvm.17538943113237448796] ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.66.0-nightly.so
#5 0x00007f4ed7983ab7 in llvm::RegAllocBase::postOptimization() ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.66.0-nightly.so
#6 0x00007f4ed797e55a in llvm::RAGreedy::runOnMachineFunction(llvm::MachineFunction&) ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.66.0-nightly.so
#7 0x00007f4ed85fcaf6 in llvm::FPPassManager::runOnFunction(llvm::Function&) ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.66.0-nightly.so
#8 0x00007f4ed85fc10f in llvm::FPPassManager::runOnModule(llvm::Module&) ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.66.0-nightly.so
#9 0x00007f4ed85fb963 in llvm::legacy::PassManagerImpl::run(llvm::Module&) ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM-15-rust-1.66.0-nightly.so
#10 0x00007f4edcef6af2 in LLVMRustWriteOutputFile ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-e91238e19c9a5d44.so
#11 0x00007f4edcef6449 in rustc_codegen_llvm::back::write::write_output_file ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-e91238e19c9a5d44.so
#12 0x00007f4edcef437a in rustc_codegen_llvm::back::write::codegen ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-e91238e19c9a5d44.so
#13 0x00007f4edcef0465 in rustc_codegen_ssa::back::write::finish_intra_module_work::<rustc_codegen_llvm::LlvmCodegenBackend> ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-e91238e19c9a5d44.so
#14 0x00007f4edceef76a in rustc_codegen_ssa::back::write::execute_work_item::<rustc_codegen_llvm::LlvmCodegenBackend> ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-e91238e19c9a5d44.so
#15 0x00007f4edceedb98 in std::sys_common::backtrace::_rust_begin_short_backtrace::<<rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa::back::write::spawn_work<rustc_codegen_llvm::LlvmCodegenBackend>::{closure#0}, ()>::{closure#0}, ()> () from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-e91238e19c9a5d44.so
#16 0x00007f4edce65480 in <std::thread::Builder::spawn_unchecked<<rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa::back::write::spawn_work<rustc_codegen_llvm::LlvmCodegenBackend>::{closure#0}, ()>::{closure#0}, ()>::{closure#1} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} ()
from /home/paul/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-e91238e19c9a5d44.so
#17 0x00007f4edab7b363 in alloc::boxed::{impl#44}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> ()
at library/alloc/src/boxed.rs:1940
#18 alloc::boxed::{impl#44}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> ()
at library/alloc/src/boxed.rs:1940
#19 std::sys::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/unix/thread.rs:108
#20 0x00007f4eda91c74d in ?? () from /usr/lib/libc.so.6
#21 0x00007f4eda99e700 in ?? () from /usr/lib/libc.so.6

[tones111]

Another data point. It looks like a workaround is to enable LTO. Adding the following to Cargo.toml appears to make the error go away when compiling with nightly-2022-09-24.

[profile.release]
lto = true

Trying to compile a debug build errors (even with LTO enabled) with

error: ran out of registers during register allocation

[m0rg-dev]

Yeah looks like #76930 is back. Can confirm nightly-2022-09-24 is where it starts breaking and that enabling LTO works as of nightly-2022-10-03.

[ids1024]

I'm seeing a Rustc segfault on the current nightly with build-std = ["core"] for target = "m68k-unknown-linux-gnu". Presumably the same issue?

Edit: GDB shows the segfault is in llvm::SelectionDAG::getNode which is different from the trace here, so perhaps a different issue.

[ids1024]

Looks like the failure with m68k is due to a llvm_unreachable(nullptr) call, which LLVM seems to use in a questionable way: https://discourse.llvm.org/t/llvm-unreachable-is-widely-misused/60587

It would be nice if LLVM could not segfault in these cases, and print a useful message even in release builds if that doesn't impact performance too much.

[Patryk27]

btw, rustc seems not to be crashing anymore on this nowadays :-)