Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Experimental Bazel mode automatically "trims" multi-configured graphs #6524

Closed
gregestren opened this issue Oct 25, 2018 · 5 comments
Closed

Experimental Bazel mode automatically "trims" multi-configured graphs #6524

gregestren opened this issue Oct 25, 2018 · 5 comments
Assignees
@sdtwigg
Labels
P2 We'll consider working on this in future. (Assignee optional) stale Issues or PRs that are stale (no activity for 30 days) team-Configurability platforms, toolchains, cquery, select(), config transitions type: feature request

Comments

@gregestren
Copy link
Contributor

Tracking issue on Bazel Configurability Roadmap

Lots of the work on that roadmap can increase build graph size unnecessarily. For example, if two Java binaries have a common data dependency on a Go binary (either directly or transitively) and they're built in the same Bazel invocation with different JDKs 1, there's no need to build the Go binary twice. But unless we make special optimizations that's exactly what will happen, because Bazel doesn't automatically distinguish that Java-related build flags don't matter for Go rules.

@serynth is prototyping work that does intelligently make these distinctions. The goal of this work is to automatically make Go rules like the above only build once and make everything generally faster and leaner as a result.

Also see "manual trimming" in #6523.

[1] This isn't really possible today because there's no interface for applying different command line flags to different targets in the same build. This example is hypothetical. But Bazel's configurability work is rapidly opening up scenarios like this, so we've got to seriously think about them.
@gregestren gregestren added P1 I'll work on this now. (Assignee required) team-Configurability platforms, toolchains, cquery, select(), config transitions labels Oct 25, 2018
bazel-io pushed a commit that referenced this issue Apr 9, 2019
@copybara-github
Split up how default flags are added in AnalysisTestCases.
62f9d39 
Not all of these flags are needed for all tests; some tests may break WITH the
excess flags.

Progress on #6524.

PiperOrigin-RevId: 242658845
bazel-io pushed a commit that referenced this issue Apr 9, 2019
@copybara-github
Add additional trimmable test rules for toolchains and platforms.
ecf2477 
Progress on #6524.

PiperOrigin-RevId: 242733707
bazel-io pushed a commit that referenced this issue Apr 9, 2019
@copybara-github
Add no-op trimming flag.
b4bc40a 
Subsequent safety changes will use this flag before the flag has any effect;
for now, it is a no-op, equivalent to specifying --experimental_dynamic_configs
equal to notrim (the default).

Progress on #6524.

PiperOrigin-RevId: 242741331
bazel-io pushed a commit that referenced this issue Apr 18, 2019
@copybara-github
Allow accessing the configuration of a dependency during resolution.
dc242d4 
This is necessary because with trimming, the configuration that a configured
target was analyzed with may not be the configuration that it was requested
with.

Progress on #6524.

PiperOrigin-RevId: 244209432
bazel-io pushed a commit that referenced this issue Apr 18, 2019
@copybara-github
Add safety checks for toolchain resolution in preparation for trimming.
375f72b 
Trimming has to make assumptions about what configuration is used in the
process of calculating toolchains. These checks enforce those assumptions.

Progress on #6524.

PiperOrigin-RevId: 244258819
bazel-io pushed a commit that referenced this issue Apr 23, 2019
@copybara-github
Add safety checks for aspects.
7c7a701 
As aspects are not currently supported in retroactive trimming mode, these
checks ensure that they are not silently executed incorrectly.

Progress on #6524.

PiperOrigin-RevId: 244946454
bazel-io pushed a commit that referenced this issue Apr 24, 2019
@copybara-github
Add a TrimmedConfigurationProgressReceiver.
ad410b1 
This will be used to monitor Skyframe changes so that the trimming cache can be
updated accordingly.

Progress on #6524.

PiperOrigin-RevId: 244956343
bazel-io pushed a commit that referenced this issue Apr 24, 2019
@copybara-github
Add safety checks for configurable attributes in trimming mode.
6e2cfe9 
Trimming does not support configurable attributes, so these checks prevent
rules with configurable attributes from silently being evaluated incorrectly.

Progress on #6524.

PiperOrigin-RevId: 244959981
bazel-io pushed a commit that referenced this issue Apr 24, 2019
@copybara-github
Add TODO about trimming Starlark build configuration values.
938db6d 
Progress on #6524.

PiperOrigin-RevId: 244960345
bazel-io pushed a commit that referenced this issue Apr 24, 2019
@copybara-github
Weave the trimming cache into Skyframe, but do not use it yet.
794a020 
Final preparations for the retroactive trimming prototype.

Progress on #6524.

PiperOrigin-RevId: 244960626
meisterT pushed a commit that referenced this issue Apr 24, 2019
@meisterT
Add safety checks for aspects.
6c224b0 
As aspects are not currently supported in retroactive trimming mode, these
checks ensure that they are not silently executed incorrectly.

Progress on #6524.

PiperOrigin-RevId: 244946454
meisterT pushed a commit that referenced this issue Apr 24, 2019
@meisterT
Add a TrimmedConfigurationProgressReceiver.
7dab827 
This will be used to monitor Skyframe changes so that the trimming cache can be
updated accordingly.

Progress on #6524.

PiperOrigin-RevId: 244956343
meisterT pushed a commit that referenced this issue Apr 24, 2019
@meisterT
Add safety checks for configurable attributes in trimming mode.
9213f95 
Trimming does not support configurable attributes, so these checks prevent
rules with configurable attributes from silently being evaluated incorrectly.

Progress on #6524.

PiperOrigin-RevId: 244959981
meisterT pushed a commit that referenced this issue Apr 24, 2019
@meisterT
Add TODO about trimming Starlark build configuration values.
090f051 
Progress on #6524.

PiperOrigin-RevId: 244960345
meisterT pushed a commit that referenced this issue Apr 24, 2019
@meisterT
Weave the trimming cache into Skyframe, but do not use it yet.
510b0b9 
Final preparations for the retroactive trimming prototype.

Progress on #6524.

PiperOrigin-RevId: 244960626
@gregestren
Copy link
Contributor Author

April '19 update:

This now exists but very much remains an experiment. You can use it with --experimental_dynamic_configs=retroactive.

@serynth is currently prioritizing doing some core memory/performance profiling and using this to automatically power Android feature flags and fast rebuilds on --test_* changes.

@gregestren gregestren mentioned this issue Jun 20, 2019
Closed
@gregestren gregestren mentioned this issue Jan 22, 2020
Closed
@katre
Copy link
Member

katre commented May 13, 2020

@gregestren Can you please provide an update on this issue?

@gregestren gregestren added P2 We'll consider working on this in future. (Assignee optional) and removed P1 I'll work on this now. (Assignee required) labels May 13, 2020
@gregestren
Copy link
Contributor Author

@sdtwigg is currently finishing up the --trim_test_configuration specialization (that has its own tracking bug), after which this will be explored more generally. He can provide further details.

@github-actions
Copy link

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 2+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

@github-actions github-actions bot added the stale Issues or PRs that are stale (no activity for 30 days) label Apr 26, 2023
@github-actions
Copy link

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please reach out to the triage team (@bazelbuild/triage). Thanks!

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sdtwigg sdtwigg

Labels
P2 We'll consider working on this in future. (Assignee optional) stale Issues or PRs that are stale (no activity for 30 days) team-Configurability platforms, toolchains, cquery, select(), config transitions type: feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@programmablereya @katre @sdtwigg @gregestren