Linker error when building dylib with "vectorcall" no_mangle fn on x86-64 Linux

[RalfJung]

Building this file...

#![feature(abi_vectorcall)]

#[no_mangle]
#[inline(never)]
pub extern "vectorcall" fn call_with_42(i: i32) {
    assert_eq!(i, 42);
}

with --crate-type=dylib leads to a linker error on my system (x86_64-unknown-linux-gnu):

$ rustc $filename --crate-type=dylib
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/self-contained:/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/self-contained:/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/self-contained:/home/r/.opam/coq-8.19/bin:/home/r/bin:/home/r/.cargo/bin:/home/r/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games" VSLANG="1033" "cc" "-Wl,--version-script=/tmp/rustc54p2UM/list" "-Wl,--no-undefined-version" "-m64" "/tmp/rustc54p2UM/symbols.o" "foreign-vectorcall.foreign_vectorcall.e458723e0d9c0914-cgu.0.rcgu.o" "foreign-vectorcall.5inwzblfweptyphk6iy8ows5f.rcgu.rmeta" "foreign-vectorcall.bq1je05mopu28e8i624qzs0l3.rcgu.o" "-Wl,--as-needed" "-Wl,-Bstatic" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-fa4c04b3e3e28f7e.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-eb97e423290e2a73.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-6a3cbaef857276ba.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-f14799023c8f2a27.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-ae67f58af1bc202d.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-07db6d7d16f60531.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-af5f9e36062cdfaa.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-f753d0635143b03f.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-02716de7358d6620.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-9099a438c5291bc6.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-4549192a05d95b2c.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-017f0d0d69a7a874.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-ad6115df66a4cafe.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-203703a910c15888.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-fbe0149ad3765332.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-d2c56345f3127d00.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-395a38b8e0851c9b.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-d453bab70303062c.rlib" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-d5e57876d7066b4c.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-B/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-B/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-B/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/r/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "libforeign_vectorcall.so" "-shared" "-Wl,-soname=libforeign_vectorcall.so" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: rust-lld: error: foreign-vectorcall.foreign_vectorcall.e458723e0d9c0914-cgu.0.rcgu.o: symbol call_with_42@@8 has undefined version 8
          collect2: error: ld returned 1 exit status

Cc #124485

[bjorn3]

I'm not sure there is anything we can do to make this work. The vectorcall calling convention requires a mangling that includes @@, but on ELF an @ indicates that the part after it is the version of the symbol, with @@ indicating the default symbol version to link against when none is specified by the caller. We should probably ban vectorcall on ELF or even any non-PE/COFF targets.

[RalfJung]

Is that specifically only affecting the vectorcall ABI? That one is still unstable so banning it on ELF targets is fairly easy.

Do C/C++ compilers support this ABI on ELF targets?

[bjorn3]

Is that specifically only affecting the vectorcall ABI?

It affects any ABI which mandates a name mangling that includes an @. On 32-bit x86 Windows it also includes fastcall and stdcall, but this mangling doesn't happen outside of Windows targets for those calling conventions.

Do C/C++ compilers support this ABI on ELF targets?

GCC does not. Clang does.

[RalfJung]

So what does clang do with those symbols on ELF targets?

[bjorn3]

Turns out with clang it works for as long as you either don't use a version script or don't pass -Wl,--no-undefined-version. Rustc has a hard dependency on using version scripts to control what symbols are exported by cdylibs. And --no-undefined-version is the default now already in upstream LLD. (I'm using an outdated one from Debian) Also rustc passes the unmangled name in the version script. Passing the mangled name as it actually appears in the object file results in the linker complaining about @ being an invalid character.