Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Access query (DepKind) metadata through fields #78452

Merged
merged 11 commits into from
Jan 8, 2021

Conversation

cjgillot
Copy link
Contributor

@cjgillot cjgillot commented Oct 27, 2020

This refactors the access to query definition metadata (attributes such as eval always, anon, has_params) and loading/forcing functions to generate a number of structs, instead of matching on the DepKind enum. This makes access to the fields cheaper to compile. Using a struct means that finding the metadata for a given query is just an offset away; previously the match may have been compiled to a jump table but likely not completely inlined as we expect here.

A previous attempt explored a similar strategy, but using trait objects in #78314 that proved less effective, likely due to higher overheads due to forcing dynamic calls and poorer cache utilization (all metadata is fairly densely packed with this PR).

@rust-highfive
Copy link
Collaborator

r? @ecstatic-morse

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 27, 2020
@jonas-schievink
Copy link
Contributor

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@bors
Copy link
Contributor

bors commented Oct 27, 2020

⌛ Trying commit 73b774e23cce033a0a9c3582da8b84bd08b2bfcc with merge a0f145ad1a2c64f79fe055f2afa324f86e69fad1...

@bors
Copy link
Contributor

bors commented Oct 27, 2020

☀️ Try build successful - checks-actions
Build commit: a0f145ad1a2c64f79fe055f2afa324f86e69fad1 (a0f145ad1a2c64f79fe055f2afa324f86e69fad1)

@rust-timer
Copy link
Collaborator

Queued a0f145ad1a2c64f79fe055f2afa324f86e69fad1 with parent 07e968b, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit (a0f145ad1a2c64f79fe055f2afa324f86e69fad1): comparison url.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. Please note that if the perf results are neutral, you should likely undo the rollup=never given below by specifying rollup- to bors.

Importantly, though, if the results of this run are non-neutral do not roll this PR up -- it will mask other regressions or improvements in the roll up.

@bors rollup=never

@cjgillot
Copy link
Contributor Author

The bug should be fixed now.

@jyn514
Copy link
Member

jyn514 commented Oct 29, 2020

@cjgillot does this need another perf run? Can you explain what went wrong before and what your fix was?

@cjgillot
Copy link
Contributor Author

Yes please.
I tried to manually inline can_reconstruct_query_key, and made copy-paste mistake reversing the whole logic.

@jyn514
Copy link
Member

jyn514 commented Oct 29, 2020

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@bors
Copy link
Contributor

bors commented Oct 29, 2020

⌛ Trying commit e2baff32118d28a653a062be815659a306d77883 with merge 6e1160604446a0c65db29e950874ac85ec019752...

@jyn514 jyn514 added the A-query-system Area: The rustc query system (https://rustc-dev-guide.rust-lang.org/query.html) label Oct 29, 2020
@bors
Copy link
Contributor

bors commented Oct 29, 2020

☀️ Try build successful - checks-actions
Build commit: 6e1160604446a0c65db29e950874ac85ec019752 (6e1160604446a0c65db29e950874ac85ec019752)

@rust-timer
Copy link
Collaborator

Queued 6e1160604446a0c65db29e950874ac85ec019752 with parent ae9731c, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit (6e1160604446a0c65db29e950874ac85ec019752): comparison url.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. Please note that if the perf results are neutral, you should likely undo the rollup=never given below by specifying rollup- to bors.

Importantly, though, if the results of this run are non-neutral do not roll this PR up -- it will mask other regressions or improvements in the roll up.

@bors rollup=never
@rustbot modify labels: +S-waiting-on-review -S-waiting-on-perf

use std::hash::Hash;

pub use rustc_query_system::dep_graph::{DepContext, DepNodeParams};

pub type DepKind = &'static DepKindStruct;
pub struct DepKindStruct {
pub(super) index: DepKindIndex,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the fields be public so you can define your own DepKind outside rustc?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole file should be moved outside rustc_middle eventually. However, there should be an exact correspondence between DepKinds, DepKindIndex and DEP_KINDS for encoding / decoding.

@jyn514
Copy link
Member

jyn514 commented Nov 1, 2020

Perf looks like mostly minor regressions, with a small improvement on match-stress-enum. rustc_incremental went down a lot, but at the cost of hurting rustc_middle.

@cjgillot
Copy link
Contributor Author

cjgillot commented Nov 1, 2020

I have not found the origin of the regression yet.
Making rustc_incremental faster makes no sense.

@cjgillot
Copy link
Contributor Author

cjgillot commented Nov 1, 2020

Removed OnDiskCache-related code, and added a few inlining directives.

@jyn514
Copy link
Member

jyn514 commented Nov 1, 2020

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@Mark-Simulacrum
Copy link
Member

@cjgillot Am I correct that you believe you've resolved all of the comments I've left?

@cjgillot
Copy link
Contributor Author

cjgillot commented Jan 7, 2021

@Mark-Simulacrum Yes. I rebased and added 3 commits to address your comments.

@Mark-Simulacrum
Copy link
Member

r=me with the last 3 commits squashed into the appropriate prior commits

@cjgillot
Copy link
Contributor Author

cjgillot commented Jan 8, 2021

@bors r=Mark-Simulacrum

@bors
Copy link
Contributor

bors commented Jan 8, 2021

📌 Commit 0f334c3 has been approved by Mark-Simulacrum

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 8, 2021
@bors
Copy link
Contributor

bors commented Jan 8, 2021

⌛ Testing commit 0f334c3 with merge 26438b4...

@bors
Copy link
Contributor

bors commented Jan 8, 2021

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 26438b4 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jan 8, 2021
@bors bors merged commit 26438b4 into rust-lang:master Jan 8, 2021
@rustbot rustbot added this to the 1.51.0 milestone Jan 8, 2021
@bors bors mentioned this pull request Jan 8, 2021
@cjgillot cjgillot deleted the ddk-struct branch January 8, 2021 21:32
is_eval_always: false,

can_reconstruct_query_key: || true,
force_from_dep_node: |_, _| false,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TraitSelect and CompileCodegenUnit used to bug! on force_from_dep_node, as with Null and CrateMetadata. Was this change intentional? Sorry if this is obvious--I haven't read the whole PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, force_from_dep_node bailed out earlier (line 167), since those DepKinds are either anonymous, or with unreconstructible key. I was aiming at keeping the exact same behaviour, but panicking should be fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha, I see now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-query-system Area: The rustc query system (https://rustc-dev-guide.rust-lang.org/query.html) merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.