toml 0.4.9 to 0.5.3 compile for a long time on 2021-10-02 nightly

[saethlin]

[dependencies]
config = "=0.9.3"

[profile.release]
panic = "abort"
codegen-units = 1

It seems like both the panic setting and codegen-units setting are required.

I don't know what is going on, but LLVM seems to think it's doing something, but it's been doing the same something for the past 30 minutes. perf top says:

  41.22%  libLLVM-13-rust-1.57.0-nightly.so  [.] llvm::getUniqueExitBlocksHelper<llvm::BasicBlock, llvm::LoopBase<llvm::BasicBlock, llvm::Loop>, llvm::LoopBase<llvm::BasicBlock, llvm::Loop>::getUniq
  36.17%  libLLVM-13-rust-1.57.0-nightly.so  [.] llvm::ScalarEvolution::forgetMemoizedResults
   7.01%  libLLVM-13-rust-1.57.0-nightly.so  [.] llvm::Instruction::getSuccessor
   5.58%  libLLVM-13-rust-1.57.0-nightly.so  [.] llvm::ScalarEvolution::getLoopDisposition
   2.63%  libLLVM-13-rust-1.57.0-nightly.so  [.] llvm::ScalarEvolution::getSCEVAtScope

rustc --version --verbose:

rustc 1.57.0-nightly (f03eb6bef 2021-10-02)
binary: rustc
commit-hash: f03eb6bef8ced8a243858b819e013b9caf83d757
commit-date: 2021-10-02
host: x86_64-unknown-linux-gnu
release: 1.57.0-nightly
LLVM version: 13.0.0

[saethlin]

Update: Compilation finished after about 40 minutes.

[ehuss]

Bisected to #88243, cc @nikic

[apiraino]

@saethlin what's the command you use to reproduce this issue? Do you add some flags, such as -Znew-llvm-pass-manager?

[saethlin]

@apiraino No flags. Just cargo build --release with the dependencies and profile.release tables in Cargo.toml.

Are you having trouble reproducing?

[apiraino]

ah yes thank you @saethlin, I didn't read carefully your configuration. yes, I can now reproduce, I believe the issue is in one of the dependencies of config=0.9.3 (dated 2019, so all dependencies are equally old or even older). I've noticed that compiling config=0.11 doesn't reproduce anymore this issue.

Assigning priority as discussed in the Zulip thread of the Prioritization Working Group.

@rustbot label -I-prioritize +P-medium +T-compiler

[saethlin]

I'm poking at this. It's out of my expertise, but maybe I'll turn up something useful.

I minimized config down to this function:

pub fn parse(text: &str) -> toml::Value {
    toml::from_str(text).unwrap()
}

So the regression is reproduced just by something in toml. It appears first in commit
f667495ae34395be9fc3e4bb8801cacf52c2253e and is removed by 7c9b0a39db0c4c235db372f88d725fae6e82889f

The older commits compile in release within a few seconds, while the more recent commits need just over a minute. I suggest investigating this based on the older toml commits in that range.

[saethlin]

--debug-pass=Details tuns up an endless stream of loop passes, which seems cruelly ironic

[2021-10-07 19:01:26.263184842] 0x7f138d061820       Executing Pass 'Canonicalize Freeze Instructions in Loops' on Loop ''...
0x7f138d5546e0         Required Analyses: Natural Loop Information, Canonicalize natural loops, Scalar Evolution Analysis, Dominator Tree Construction
0x7f138d5546e0         Preserved Analyses: Canonicalize natural loops, Natural Loop Information, Scalar Evolution Analysis, Dominator Tree Construction
 -*- 'Canonicalize Freeze Instructions in Loops' is the last user of following pass instances. Free these instances
[2021-10-07 19:01:26.263400523] 0x7f138d061820        Freeing Pass 'Canonicalize Freeze Instructions in Loops' on Loop ''...
[2021-10-07 19:01:26.263403573] 0x7f138d061820       Executing Pass 'Induction Variable Users' on Loop ''...
0x7f138d407180         Required Analyses: Assumption Cache Tracker, Natural Loop Information, Dominator Tree Construction, Scalar Evolution Analysis
[2021-10-07 19:01:26.330309621] 0x7f138d061820       Executing Pass 'Loop Strength Reduction' on Loop ''...
0x7f138d554420         Required Analyses: Natural Loop Information, Canonicalize natural loops, Dominator Tree Construction, Scalar Evolution Analysis, Assumption Cache Tracker, Target Library Information, Induction Variable Users, Target Transform Information
[2021-10-07 19:01:26.330791412] 0x7f138d061820       Made Modification 'Loop Strength Reduction' on Loop '<unnamed loop>'...
0x7f138d554420         Preserved Analyses: Canonicalize natural loops, Natural Loop Information, Dominator Tree Construction, Scalar Evolution Analysis, Induction Variable Users, Memory SSA
 -*- 'Loop Strength Reduction' is the last user of following pass instances. Free these instances
[2021-10-07 19:01:26.330804321] 0x7f138d061820        Freeing Pass 'Induction Variable Users' on Loop ''...
[2021-10-07 19:01:26.330808321] 0x7f138d061820        Freeing Pass 'Loop Strength Reduction' on Loop ''...
[2021-10-07 19:01:26.330812571] 0x7f138d061820       Executing Pass 'Canonicalize Freeze Instructions in Loops' on Loop ''...

[nikic]

Looks like LLVM manages to blow up _ZN126_$LT$$LT$toml..value..Value$u20$as$u20$serde..de..Deserialize$GT$..deserialize..ValueVisitor$u20$as$u20$serde..de..Visitor$GT$9visit_map17h0937f33df0be8d95E to approximately 750k instructions.

[nikic]

Adding some size instrumentation, we end up running the cgscc pipeline on this function four times, which includes inlining and the function simplification pipeline. On each inline, the size jumps from 325 originally to 16069 to 103390 to 778219 instructions. This is not due to devirt iteration. I can avoid the last one by dropping the UpdatedC reprocessing. Something is going catastrophically wrong here in the interaction between the CGSCC pass manager and the inliner.

[nikic]

I think the issue here is recursive drop glue that gets recursively inlined. This looks similar to the report at https://bugs.llvm.org/show_bug.cgi?id=45253.

[Mark-Simulacrum]

@nikic do we think we can push the patch for fixing this forward on the LLVM side somehow? It seems desirable to fix this as soon as we can, though it likely won't make the next release given the short timeline...

[nikic]

@Mark-Simulacrum I don't have much hope for a timely upstream fix. We can cherry-pick https://reviews.llvm.org/D98481. It has minor perf impact per #89830.

[adrian17]

For the record, this and #89647 still appear to hang and consume lots of memory. (I was quietly hoping #92110 would improve it)

Is this still considered a NewPM blocker?

[saethlin]

Checking in, the problem documented in this issue no longer reproduces on nightly or stable (1.58.1).

But the example documented in #89647 still seems to be a problem.

[nikic]

Closing this one as the specific case doesn't reproduce anymore, and we have other issues tracking similar problems.