Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incremental compilation bug with async fn method capturing multiple lifetimes #98890

Closed
danielhenrymantilla opened this issue Jul 4, 2022 · 10 comments · Fixed by #98959
Closed
Labels
C-bug Category: This is a bug. P-critical Critical priority regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@danielhenrymantilla
Copy link
Contributor

Regression since 1.62.0

The following diff:

  struct Foo;

  impl Foo {
      async fn f(&self, _: &&()) -> &() {
          &()
      }
  }

+ enum Bar {}

if done in between cargo checks, makes the second check fail with:

error: lifetime may not live long enough
 --> src/lib.rs:5:43
  |
5 |           async fn f(&self, _: &&()) -> &() {
  |  ____________________-__________-___________^
  | |                    |          |
  | |                    |          let's call the lifetime of this reference `'2`
  | |                    let's call the lifetime of this reference `'1`
6 | |             &()
7 | |         }
  | |_________^ associated function was supposed to return data with lifetime `'2` but it is returning data with lifetime `'1`
  • Aside: that error is printed twice, which is also curious.

To reproduce, the following snippet fails on the Playground:

fn main(){let _=std::process::Command::new("/bin/bash").args(&["-c", concat!(r##"{
set -euo pipefail
cd $(mktemp -d)


cargo init -q --name example --lib

cat> src/lib.rs <<'EOF'
    #![allow(unused)]
    struct Foo;
    
    impl Foo {
        async fn f(&self, _: &&()) -> &() {
            &()
        }
    }
EOF

(set -x
    cargo c -q
    { set +x; echo 'enum Bar {}'; } 2>/dev/null | tee -a src/lib.rs
    cargo c -q
)


echo ✅
} 2>&1"##)]).status();}

I've reported it over #96655 to avoid defining a duplicate issue, but now that I've noticed it's a regression, I prefer to risk it being a dup than it going unnoticed.

I've also mentioned this over: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/incremental.20bug.20with.20async.20fns.20and.20lifetimes


Context

This is causing major breakage with users of async-graphql who have switched to Rust 1.62.0 or up, as showcased by:


@rustbot modify labels: +regression-from-stable-to-stable -regression-untriaged

@danielhenrymantilla danielhenrymantilla added C-bug Category: This is a bug. regression-untriaged Untriaged performance or correctness regression. labels Jul 4, 2022
@rustbot rustbot added I-prioritize Issue: Indicates that prioritization has been requested for this issue. regression-from-stable-to-stable Performance or correctness regression from one stable version to another. and removed regression-untriaged Untriaged performance or correctness regression. labels Jul 4, 2022
@michaelwoerister
Copy link
Member

The bug also reproduces when the enum Bar {} is added above the impl, but only when that does not change the spans of the impl. It also reproduces when the enum Bar {} is add in a different file, or when a struct or an fn is added instead.

This suggests to me that this is somehow related to the overall set of DefIds changing.

It does not reproduce when the function is non-async.

@rust-lang/wg-incr-comp, any ideas? Where there any big refactorings during that time frame that could be the cause of this?

@michaelwoerister
Copy link
Member

michaelwoerister commented Jul 5, 2022

Maybe something related to generators / async functions generates numeric IDs based raw DefIndex values which are then not hashed in a stable way?

@lqd
Copy link
Member

lqd commented Jul 5, 2022

Bisection seems to point to #91557

@michaelwoerister
Copy link
Member

That does sound related. @cjgillot might have an idea.

@apiraino apiraino added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jul 6, 2022
@apiraino
Copy link
Contributor

apiraino commented Jul 6, 2022

WG-prioritization assigning priority (Zulip discussion).

@rustbot label -I-prioritize +P-critical

@rustbot rustbot added P-critical Critical priority and removed I-prioritize Issue: Indicates that prioritization has been requested for this issue. labels Jul 6, 2022
@bors bors closed this as completed in 3dcb616 Jul 6, 2022
@michaelwoerister
Copy link
Member

Let's keep this open until the backporting situation of the fix is clarified.

@cuviper
Copy link
Member

cuviper commented Jul 13, 2022

#98959 was backported to 1.63.0-beta in #99098 -- ready to close?

@iliana
Copy link

iliana commented Jul 13, 2022

Is this worth a stable backport? The experience of developing async code with this issue isn’t great.

@cuviper
Copy link
Member

cuviper commented Jul 13, 2022

I tried a stable backport locally just to see, but it's non-trivial -- #98959 builds on changes from #97415 and #97023 at least, but that's as far as I investigated before giving up. Maybe a more targeted fix for 1.62.0 could be developed though.

bors bot added a commit to embassy-rs/embassy that referenced this issue Jul 13, 2022
865: Update Rust nightly. r=Dirbaio a=Dirbaio

Includes fix for rust-lang/rust#98890 which has been annoying me in a few projects.

bors r+

Co-authored-by: Dario Nieuwenhuis <[email protected]>
Dylan-DPC added a commit to Dylan-DPC/rust that referenced this issue Jul 18, 2022
…and-map, r=nagisa

Remove the unused StableSet and StableMap types from rustc_data_structures.

The current implementation is not "stable" in the same sense that `HashStable` and `StableHasher` are stable, i.e. across compilation sessions. So, in my opinion, it's better to remove those types (which are basically unused anyway) than to give the wrong impression that these are safe for incr. comp.

I plan to provide new "stable" collection types soon that can be used to replace `FxHashMap` and `FxHashSet` in query results (see [draft](michaelwoerister@69d03ac)). It's unsound that `HashMap` and `HashSet` implement `HashStable` (see rust-lang#98890 for a recent P-critical bug caused by this) -- so we should make some progress there.
@cjgillot
Copy link
Contributor

Backported to stable.

wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this issue Jul 20, 2022
Pkgsrc changes:
 * none.

Upstream changes:

Version 1.62.1 (2022-07-19)
==========================

Rust 1.62.1 addresses a few recent regressions in the compiler and standard
library, and also mitigates a CPU vulnerability on Intel SGX.

* [The compiler fixed unsound function coercions involving `impl
  Trait` return types.][98608]
* [The compiler fixed an incremental compilation bug with `async
  fn` lifetimes.][98890]
* [Windows added a fallback for overlapped I/O in synchronous reads
  and writes.][98950]
* [The `x86_64-fortanix-unknown-sgx` target added a mitigation for the
  MMIO stale data vulnerability][98126], advisory [INTEL-SA-00615].

[98608]: rust-lang/rust#98608
[98890]: rust-lang/rust#98890
[98950]: rust-lang/rust#98950
[98126]: rust-lang/rust#98126
[INTEL-SA-00615]: https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00615.html
bors added a commit to rust-lang-ci/rust that referenced this issue Jul 20, 2022
…d-map, r=nagisa

Remove the unused StableSet and StableMap types from rustc_data_structures.

The current implementation is not "stable" in the same sense that `HashStable` and `StableHasher` are stable, i.e. across compilation sessions. So, in my opinion, it's better to remove those types (which are basically unused anyway) than to give the wrong impression that these are safe for incr. comp.

I plan to provide new "stable" collection types soon that can be used to replace `FxHashMap` and `FxHashSet` in query results (see [draft](michaelwoerister@69d03ac)). It's unsound that `HashMap` and `HashSet` implement `HashStable` (see rust-lang#98890 for a recent P-critical bug caused by this) -- so we should make some progress there.
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Aug 31, 2022
Pkgsrc changes:

 * Bump required GCC to 7 (same as LLVM) to avoid ABI issues
   Fixes native i386 and powerpc 8.x build w/pkgsrc LLVM 14
 * Bump available bootstraps to 1.61.0.
 * Also unlimit stacksize
 * Sync patches over from wip/rust
 * Adjust line number in patches which had non-zero offsets.
 * no longer pass -I/usr/pkg/include through via gcc-wrap script
   when building natively.  Attempt at fixing version skew with curl
   package vs. internal version of curl (may not work...)
 * The NetBSD bootstraps now use .xz compression.
 * Use mk/atomic64.mk.  Still have conditional for libatomic-links.
 * Default to using the internal LLVM when cross-building.


Upstream changes:

Version 1.62.1 (2022-07-19)
==========================

Rust 1.62.1 addresses a few recent regressions in the compiler and standard
library, and also mitigates a CPU vulnerability on Intel SGX.

* [The compiler fixed unsound function coercions involving `impl
  Trait` return types.][98608]
* [The compiler fixed an incremental compilation bug with `async
  fn` lifetimes.][98890]
* [Windows added a fallback for overlapped I/O in synchronous reads
  and writes.][98950]
* [The `x86_64-fortanix-unknown-sgx` target added a mitigation for the
  MMIO stale data vulnerability][98126], advisory [INTEL-SA-00615].

[98608]: rust-lang/rust#98608
[98890]: rust-lang/rust#98890
[98950]: rust-lang/rust#98950
[98126]: rust-lang/rust#98126
[INTEL-SA-00615]: https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00615.html


Version 1.62.0 (2022-06-30)
==========================

Language
--------

- [Stabilize `#[derive(Default)]` on enums with a `#[default]` variant][94457]
- [Stop validating some checks in dead code after functions with
  uninhabited return types][93313]
- [Fix constants not getting dropped if part of a diverging expression][94775]
- [Support unit struct/enum variant in destructuring assignment][95380]
- [Remove mutable_borrow_reservation_conflict lint and allow the
  code pattern][96268]

Compiler
--------

- [linker: Stop using whole-archive on dependencies of dylibs][96436]
- [Make `unaligned_references` lint deny-by-default][95372]
  This lint is also a future compatibility lint, and is expected to eventually
  become a hard error.
- [Only add codegen backend to dep info if -Zbinary-dep-depinfo is used][93969]
- [Reject `#[thread_local]` attribute on non-static items][95006]
- [Add tier 3 `aarch64-pc-windows-gnullvm` and `x86_64-pc-windows-gnullvm`
  targets\*][94872]
- [Implement a lint to warn about unused macro rules][96150]
- [Promote `x86_64-unknown-none` target to Tier 2\*][95705]

\* Refer to Rust's [platform support page][platform-support-doc] for more
   information on Rust's tiered platform support.

Libraries
---------

- [Move `CStr` to libcore, and `CString` to liballoc][94079]
- [Windows: Use a pipe relay for chaining pipes][95841]
- [Replace Linux Mutex and Condvar with futex based ones.][95035]
- [Replace RwLock by a futex based one on Linux][95801]
- [std: directly use pthread in UNIX parker implementation][96393]

Stabilized APIs
---------------

- [`bool::then_some`]
- [`f32::total_cmp`]
- [`f64::total_cmp`]
- [`Stdin::lines`]
- [`windows::CommandExt::raw_arg`]
- [`impl<T: Default> Default for AssertUnwindSafe<T>`]
- [`From<Rc<str>> for Rc<[u8]>`][rc-u8-from-str]
- [`From<Arc<str>> for Arc<[u8]>`][arc-u8-from-str]
- [`FusedIterator for EncodeWide`]
- [RDM intrinsics on aarch64][stdarch/1285]

Clippy
------

- [Create clippy lint against unexpectedly late drop for temporaries
  in match scrutinee expressions][94206]

Cargo
-----

- Added the `cargo add` command for adding dependencies to `Cargo.toml` from
  the command-line.
  [docs](https://doc.rust-lang.org/nightly/cargo/commands/cargo-add.html)
- Package ID specs now support `name@version` syntax in addition to the
  previous `name:version` to align with the behavior in `cargo add` and other
  tools. `cargo install` and `cargo yank` also now support this syntax so the
  version does not need to passed as a separate flag.
- The `git` and `registry` directories in Cargo's home directory (usually
  `~/.cargo`) are now marked as cache directories so that they are not
  included in backups or content indexing (on Windows).
- Added automatic `@` argfile support, which will use "response files" if the
  command-line to `rustc` exceeds the operating system's limit.

Compatibility Notes
-------------------

- `cargo test` now passes `--target` to `rustdoc` if the specified target is
  the same as the host target.
  [#10594](rust-lang/cargo#10594)
- [rustdoc: Remove .woff font files][96279]
- [Enforce Copy bounds for repeat elements while considering lifetimes][95819]

Internal Changes
----------------

- [Unify ReentrantMutex implementations across all platforms][96042]

These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

[93313]: rust-lang/rust#93313
[93969]: rust-lang/rust#93969
[94079]: rust-lang/rust#94079
[94206]: rust-lang/rust#94206
[94457]: rust-lang/rust#94457
[94775]: rust-lang/rust#94775
[94872]: rust-lang/rust#94872
[95006]: rust-lang/rust#95006
[95035]: rust-lang/rust#95035
[95372]: rust-lang/rust#95372
[95380]: rust-lang/rust#95380
[95431]: rust-lang/rust#95431
[95705]: rust-lang/rust#95705
[95801]: rust-lang/rust#95801
[95819]: rust-lang/rust#95819
[95841]: rust-lang/rust#95841
[96042]: rust-lang/rust#96042
[96150]: rust-lang/rust#96150
[96268]: rust-lang/rust#96268
[96279]: rust-lang/rust#96279
[96393]: rust-lang/rust#96393
[96436]: rust-lang/rust#96436
[96557]: rust-lang/rust#96557

[`bool::then_some`]: https://doc.rust-lang.org/stable/std/primitive.bool.html#method.then_some
[`f32::total_cmp`]: https://doc.rust-lang.org/stable/std/primitive.f32.html#method.total_cmp
[`f64::total_cmp`]: https://doc.rust-lang.org/stable/std/primitive.f64.html#method.total_cmp
[`Stdin::lines`]: https://doc.rust-lang.org/stable/std/io/struct.Stdin.html#method.lines
[`impl<T: Default> Default for AssertUnwindSafe<T>`]: https://doc.rust-lang.org/stable/std/panic/struct.AssertUnwindSafe.html#impl-Default
[rc-u8-from-str]: https://doc.rust-lang.org/stable/std/rc/struct.Rc.html#impl-From%3CRc%3Cstr%3E%3E
[arc-u8-from-str]: https://doc.rust-lang.org/stable/std/sync/struct.Arc.html#impl-From%3CArc%3Cstr%3E%3E
[stdarch/1285]: rust-lang/stdarch#1285
[`windows::CommandExt::raw_arg`]: https://doc.rust-lang.org/stable/std/os/windows/process/trait.CommandExt.html#tymethod.raw_arg
[`FusedIterator for EncodeWide`]: https://doc.rust-lang.org/stable/std/os/windows/ffi/struct.EncodeWide.html#impl-FusedIterator


Version 1.61.0 (2022-05-19)
==========================

Language
--------

- [`const fn` signatures can now include generic trait bounds][93827]
- [`const fn` signatures can now use `impl Trait` in argument and return
  position][93827]
- [Function pointers can now be created, cast, and passed around in a
  `const fn`][93827]
- [Recursive calls can now set the value of a function's opaque
  `impl Trait` return type][94081]

Compiler
--------

- [Linking modifier syntax in `#[link]` attributes and on the command
  line, as well as the `whole-archive` modifier specifically, are now
  supported][93901]
- [The `char` type is now described as UTF-32 in debuginfo][89887]
- The [`#[target_feature]`][target_feature] attribute
  [can now be used with aarch64 features][90621]
- X86 [`#[target_feature = "adx"]` is now stable][93745]

Libraries
---------

- [`ManuallyDrop<T>` is now documented to have the same layout as `T`][88375]
- [`#[ignore = "#"]` messages are printed when running tests][92714]
- [Consistently show absent stdio handles on Windows as NULL handles][93263]
- [Make `std::io::stdio::lock()` return `'static` handles.][93965]
  Previously, the creation of locked handles to stdin/stdout/stderr would
  borrow the handles being locked, which prevented writing
  `let out = std::io::stdout().lock();` because `out` would outlive
  the return value of `stdout()`.
  Such code now works, eliminating a common pitfall that affected many
  Rust users.
- [`Vec::from_raw_parts` is now less restrictive about its inputs][95016]
- [`std::thread::available_parallelism` now takes cgroup quotas into
  account.][92697] Since `available_parallelism` is often used to create a
  thread pool for parallel computation, which may be CPU-bound for
  performance, `available_parallelism` will return a value consistent with
  the ability to use that many threads continuously, if possible.
  For instance, in a container with 8 virtual CPUs but quotas only allowing
  for 50% usage, `available_parallelism` will return 4.

Stabilized APIs
---------------

- [`Pin::static_mut`]
- [`Pin::static_ref`]
- [`Vec::retain_mut`]
- [`VecDeque::retain_mut`]
- [`Write` for `Cursor<[u8; N]>`][cursor-write-array]
- [`std::os::unix::net::SocketAddr::from_pathname`]
- [`std::process::ExitCode`] and [`std::process::Termination`].
  The stabilization of these two API s now makes it possible for
  programs to return errors from `main` with custom exit codes.
- [`std::thread::JoinHandle::is_finished`]

These APIs are now usable in const contexts:

- [`<*const T>::offset` and `<*mut T>::offset`][ptr-offset]
- [`<*const T>::wrapping_offset` and `<*mut T>::wrapping_offset`]
  [ptr-wrapping_offset]
- [`<*const T>::add` and `<*mut T>::add`][ptr-add]
- [`<*const T>::sub` and `<*mut T>::sub`][ptr-sub]
- [`<*const T>::wrapping_add` and `<*mut T>::wrapping_add`][ptr-wrapping_add]
- [`<*const T>::wrapping_sub` and `<*mut T>::wrapping_sub`][ptr-wrapping_sub]
- [`<[T]>::as_mut_ptr`][slice-as_mut_ptr]
- [`<[T]>::as_ptr_range`][slice-as_ptr_range]
- [`<[T]>::as_mut_ptr_range`][slice-as_mut_ptr_range]

Cargo
-----

No feature changes, but see compatibility notes.

Compatibility Notes
-------------------

- Previously native static libraries were linked as `whole-archive` in
  some cases, but now rustc tries not to use `whole-archive` unless
  explicitly requested. This [change][93901] may result in linking errors
  in some cases. To fix such errors, native libraries linked from the
  command line, build scripts, or [`#[link]` attributes][link-attr] need to
  - (more common) either be reordered to respect dependencies between them
    (if `a` depends on `b` then `a` should go first and `b` second)
  - (less common) or be updated to use the [`+whole-archive`] modifier.
- [Catching a second unwind from FFI code while cleaning up from a Rust
  panic now causes the process to abort][92911]
- [Proc macros no longer see `ident` matchers wrapped in groups][92472]
- [The number of `#` in `r#` raw string literals is now required to be
  less than 256][95251]
- [When checking that a dyn type satisfies a trait bound, supertrait
  bounds are now enforced][92285]
- [`cargo vendor` now only accepts one value for each `--sync` flag]
  [cargo/10448]
- [`cfg` predicates in `all()` and `any()` are always evaluated to detect
  errors, instead of short-circuiting.][94295] The compatibility
  considerations here arise in nightly-only code that used the
  short-circuiting behavior of `all` to write something like
  `cfg(all(feature = "nightly", syntax-requiring-nightly))`, which
  will now fail to compile. Instead, use either `cfg_attr(feature
  = "nightly", ...)` or nested uses of `cfg`.
- [bootstrap: static-libstdcpp is now enabled by default, and can
  now be disabled when llvm-tools is enabled][94832]

Internal Changes
----------------

These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

- [debuginfo: Refactor debuginfo generation for types][94261]
- [Remove the everybody loops pass][93913]

[88375]: rust-lang/rust#88375
[89887]: rust-lang/rust#89887
[90621]: rust-lang/rust#90621
[92285]: rust-lang/rust#92285
[92472]: rust-lang/rust#92472
[92697]: rust-lang/rust#92697
[92714]: rust-lang/rust#92714
[92911]: rust-lang/rust#92911
[93263]: rust-lang/rust#93263
[93745]: rust-lang/rust#93745
[93827]: rust-lang/rust#93827
[93901]: rust-lang/rust#93901
[93913]: rust-lang/rust#93913
[93965]: rust-lang/rust#93965
[94081]: rust-lang/rust#94081
[94261]: rust-lang/rust#94261
[94295]: rust-lang/rust#94295
[94832]: rust-lang/rust#94832
[95016]: rust-lang/rust#95016
[95251]: rust-lang/rust#95251
[`+whole-archive`]: https://doc.rust-lang.org/stable/rustc/command-line-arguments.html#linking-modifiers-whole-archive
[`Pin::static_mut`]: https://doc.rust-lang.org/stable/std/pin/struct.Pin.html#method.static_mut
[`Pin::static_ref`]: https://doc.rust-lang.org/stable/std/pin/struct.Pin.html#method.static_ref
[`Vec::retain_mut`]: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.retain_mut
[`VecDeque::retain_mut`]: https://doc.rust-lang.org/stable/std/collections/struct.VecDeque.html#method.retain_mut
[`std::os::unix::net::SocketAddr::from_pathname`]: https://doc.rust-lang.org/stable/std/os/unix/net/struct.SocketAddr.html#method.from_pathname
[`std::process::ExitCode`]: https://doc.rust-lang.org/stable/std/process/struct.ExitCode.html
[`std::process::Termination`]: https://doc.rust-lang.org/stable/std/process/trait.Termination.html
[`std::thread::JoinHandle::is_finished`]: https://doc.rust-lang.org/stable/std/thread/struct.JoinHandle.html#method.is_finished
[cargo/10448]: rust-lang/cargo#10448
[cursor-write-array]: https://doc.rust-lang.org/stable/std/io/struct.Cursor.html#impl-Write-4
[link-attr]: https://doc.rust-lang.org/stable/reference/items/external-blocks.html#the-link-attribute
[ptr-add]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.add
[ptr-offset]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset
[ptr-sub]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.sub
[ptr-wrapping_add]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.wrapping_add
[ptr-wrapping_offset]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.wrapping_offset
[ptr-wrapping_sub]: https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.wrapping_sub
[slice-as_mut_ptr]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.as_mut_ptr
[slice-as_mut_ptr_range]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.as_mut_ptr_range
[slice-as_ptr_range]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.as_ptr_range
[target_feature]: https://doc.rust-lang.org/reference/attributes/codegen.html#the-target_feature-attribute
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. P-critical Critical priority regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
8 participants