Rustbuild doesn't handle multiple versions of crates.io crates in use #42261
Labels
T-bootstrap
Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Comments
alexcrichton
added
the
T-bootstrap
alexcrichton
added a commit
to alexcrichton/rust
that referenced
this issue
May 28, 2017
After compiling a project (e.g. libstd, libtest, or librustc) rustbuild needs to copy over all artifacts into the sysroot of the compiler it's assembling. Unfortunately rustbuild doesn't know precisely what files to copy! Today it has a heuristic where it just looks at the most recent version of all files that look like rlibs/dylibs and copies those over. This unfortunately leads to bugs with different versions of the same craet as seen in rust-lang#42261. This commit updates rustbuild's strategy of copying artifacts to work off the list of artifacts produced by `cargo build --message-format=json`. The build system will now parse json messages coming out of Cargo to watch for files being generated, and then it'll only copy over those precise files. Note that there's still a bit of weird logic where Cargo prints that it's creating `libstd.rlib` where we actually want `libstd-xxxxx.rlib`, so we still do a bit of "most recent file" probing for those. This commit should take care of the crates.io dependency issues, however, as they're all copied over precisely. Closes rust-lang#42261
bors
added a commit
that referenced
this issue
May 29, 2017
rustbuild: Fix copying duplicate crates into the sysroot After compiling a project (e.g. libstd, libtest, or librustc) rustbuild needs to copy over all artifacts into the sysroot of the compiler it's assembling. Unfortunately rustbuild doesn't know precisely what files to copy! Today it has a heuristic where it just looks at the most recent version of all files that look like rlibs/dylibs and copies those over. This unfortunately leads to bugs with different versions of the same craet as seen in #42261. This commit updates rustbuild's strategy of copying artifacts to work off the list of artifacts produced by `cargo build --message-format=json`. The build system will now parse json messages coming out of Cargo to watch for files being generated, and then it'll only copy over those precise files. Note that there's still a bit of weird logic where Cargo prints that it's creating `libstd.rlib` where we actually want `libstd-xxxxx.rlib`, so we still do a bit of "most recent file" probing for those. This commit should take care of the crates.io dependency issues, however, as they're all copied over precisely. Closes #42261
alexcrichton
added a commit
to alexcrichton/rust
that referenced
this issue
May 30, 2017
After compiling a project (e.g. libstd, libtest, or librustc) rustbuild needs to copy over all artifacts into the sysroot of the compiler it's assembling. Unfortunately rustbuild doesn't know precisely what files to copy! Today it has a heuristic where it just looks at the most recent version of all files that look like rlibs/dylibs and copies those over. This unfortunately leads to bugs with different versions of the same craet as seen in rust-lang#42261. This commit updates rustbuild's strategy of copying artifacts to work off the list of artifacts produced by `cargo build --message-format=json`. The build system will now parse json messages coming out of Cargo to watch for files being generated, and then it'll only copy over those precise files. Note that there's still a bit of weird logic where Cargo prints that it's creating `libstd.rlib` where we actually want `libstd-xxxxx.rlib`, so we still do a bit of "most recent file" probing for those. This commit should take care of the crates.io dependency issues, however, as they're all copied over precisely. Closes rust-lang#42261
bors
added a commit
that referenced
this issue
Jun 1, 2017
rustbuild: Fix copying duplicate crates into the sysroot After compiling a project (e.g. libstd, libtest, or librustc) rustbuild needs to copy over all artifacts into the sysroot of the compiler it's assembling. Unfortunately rustbuild doesn't know precisely what files to copy! Today it has a heuristic where it just looks at the most recent version of all files that look like rlibs/dylibs and copies those over. This unfortunately leads to bugs with different versions of the same crate as seen in #42261. This commit updates rustbuild's strategy of copying artifacts to work off the list of artifacts produced by `cargo build --message-format=json`. The build system will now parse json messages coming out of Cargo to watch for files being generated, and then it'll only copy over those precise files. Note that there's still a bit of weird logic where Cargo prints that it's creating `libstd.rlib` where we actually want `libstd-xxxxx.rlib`, so we still do a bit of "most recent file" probing for those. This commit should take care of the crates.io dependency issues, however, as they're all copied over precisely. Closes #42261
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Rustbuild compiles the compiler in a particular location and is then responsible for copying artifacts over to the actual structure the compiler is expecting (e.g. rustbuild assembles a sysroot). In order to handle stale folders rustbuild copies only the most recent version of each rlib, deduplicated by hash.
This pretty obviously, though, doesn't work with multiple versions of the same library! We should probably call Cargo with
--message-format=jsonand learn about compiled artifacts, that way we can copy precisely what we need to into the sysroot.The text was updated successfully, but these errors were encountered: