Better Toolchain Error and Help Messages

[RefinedSoftwareLLC]

Problem

Feature to help debug cargo run failing on first-use or changed computer setup (automatic OS update):
New users should not be expected to know rustc, linkers, llvm, cc, nor ld to use cargo run.
New users should not be expected to know which packages have which .o files for which archs.

Current message

$ cargo run
(

error: linking with `cc` failed: exit status: 1
  |

...45 lines of gibberish...

  = note: ld: error: cannot open Scrt1.o: No such file or directory
          ld: error: cannot open crti.o: No such file or directory
          ld: error: unable to find library -lutil
          ld: error: unable to find library -lrt
          ld: error: unable to find library -lpthread
          ld: error: unable to find library -lm
          ld: error: unable to find library -ldl
          ld: error: unable to find library -lc
          ld: error: cannot open crtn.o: No such file or directory
          collect2: error: ld returned 1 exit status

)
...repeated 12 more times.

Proposed Solution

Desired message

$ cargo run

error[E99999]: toolchain `nightly-i686-unknown-linux-gnu` failed linking using `cc`: exit status: 1
  |
  = note: unable to find glibc-devel (crti.o, crtn.o, and Scrt1.o)
  = help: to install glibc-devel for i686, try `rpm-ostree install glibc-devel.i686`
  = help: for native toolchain, try `rustup default nightly`
  = note: visit <https://doc.rust-lang.org/nightly/rustc/platform-support/{platform-page-with-failed-requirements}.html> for more details

For more information about this error, try `rustc --explain E99999`.
error: could not compile `name` (build script) due to 1 previous error

1. A first pass improvement on this is a low hanging fruit, but a robust solution will take a lot of work and tests per target-triple os pair.

2. The platform link https://doc.rust-lang.org/nightly/rustc/platform-support/{platform-with-failed-requirements-page}.html
   should display if the toolchain has detected failed requirements or complex undetectable requirements https://doc.rust-lang.org/nightly/rustc/platform-support/apple-darwin.html

3. Take into account if current project is stable or unstable, but default to unstable if unknown (for compatibility).

4. Take into account best/default available package manager with sane prioritizing: nix else rpm-ostree else dnf/apt-get else ...etc.

5. Detect which packages are needed and what labels to use that aren't fixed to a version. (glibc-devel.i686 instead of glibc-devel-2.39-22.fc40.i686, this allows the package manager to handle version requirements.) Show "help: " message(s) recommending needed package(s).

6. Do not show cc and ld errors if the issue is just a tool-chain dependency isn't installed.

7. I also installed lld.i686 at some point and I am not sure if rpm-ostree install glibc-devel.i686 lld.i686 is more accurate.

Notes

Versions

$ cargo --version

cargo 1.83.0-nightly (a9a418d1a 2024-09-15)

[epage]

Note that cc messages aren't coming from Cargo and Cargo has no way of knowing or reporting on this. We are very much interested in the idea of a common crate for managing the build of -sys crates which could centrally own good error messages like this.

CC @weihanglo for input in any designs you are working with.

[RefinedSoftwareLLC]

@epage Thank you for your quick response and offering potential ways forward.

Doesn't the toolchain nightly-i686-unknown-linux-gnu on a x86_64 computer always need i686's crti.o, crtn.o, and Scrt1.o? Even if it is based on which linker you have configured, each native-triple + toolchain + linker set should know in advance which .o/.dll are needed (for at least rust-std). Which can be checked for, either before running the linker, or after the linker returns a non-zero status code. It would be a stop-gap with higher maintenance, but doable with a call to arms for people to submit which packages are needed for their OS + toolchain-triple.

Improving this would also help rust gain new users, as devs learning rust to try out bevy have gotten stuck on linker errors, unable to even run hello world.

[epage]

so if I understand correctly, you are suggesting we pre-emptively check to see if a known set of libraries for a platform-target triple are present before compiling?

For myself, I worry about making that list accurate and our ability to look them up considering all of the shenanigans users can do with their builds.

[RefinedSoftwareLLC]

I understand build times are already a pain point, so the fastest way is to only check after cc/ld does not return status 0.

An alternative option would be to preemptively check in the PATH list that cc/ld will be passed, for any needed linker files that are not bundled/managed with rust. I am not sure if package managers can answer faster then the path search option (definitely not all of them). (In my experience, package managers are infuriatingly slow.) But you would always need to directly check the package manager before showing the developer a "help: try package manager install package" message.

[RefinedSoftwareLLC]

I think there is a huge benefit to sane error messages for new users not doing any shenanigans (and for existing users where their OS update changed something). The example above is a >750 line confusing error message that can be detected and replaced with a useful 3 line help message covering 2 different solutions (install these dependencies or change to this native toolchain).

[RefinedSoftwareLLC]

I think it would be beneficial for rustup update and cargo update to do these checks as well and maybe even ask to fix them.

[weihanglo]

Thanks for the suggestion. It definitely could improve the user experience for newly onboard users, yet I am not sure how feasible at this moment can Cargo provide a better message.

Indeed, Cargo capture diagnostics from the compiler and re-render it to the console. However, the linking error was emitted by the compiler from here, which doesn't seem to have an error code to identify. Cargo may need to guess from the error message body for the root cause. Not too ideal and may be false. Cargo may need to re-implement the logic of finding linker/cc/compiler-runtime-lib, which could be intimidating, especially when considering different versions of different C compiler tooclhains like LLVM, GCC (I've bumped into a bug that a version of LLVM doesn't respect --gcc-toolchain flag well on some certain platforms).

Cargo could potentially collapse the 750-line error messages into one concise messsage, and redirect the original message to a file. However, this may have an impact on build logs in CI, as people may not have way to access and read the redirected log. This will be a problem because Cargo's guess might not always be correct.

That being said, I am not turning this down. It just has some technical issue needing more design and exploration.