Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add always-install-via-wheel feature flag #9778

Closed

Conversation

sbidoul
Copy link
Member

@sbidoul sbidoul commented Apr 5, 2021

Towards #9769
Includes #9776

Add the always-install-via-wheel feature flag. When enabled, it has the following effects:

  • Installation of source distributions is always done by first building a wheel then installing from it (assuming the wheel package is installed).
  • --no-binary does not imply setup.py install anymore.
  • --build-option and --global-option are passed to setup.py bdist_wheel during installation, as it was already done when using pip wheel.
  • --install-option is ignored (because there is no setup.py install involved)

Although the code change is small the implications are many, because so many things are entangled with --no-binary / FormatControl. I believe this disentangling will ultimately be good both for the code base and usability.

One thing that I might consider pushing further (in another PR) is the wheel cache control: now that a wheel can be cached as the result of an installation that includes --global-option or --build-option we should probably include those options as part of the cache key.

@sbidoul sbidoul force-pushed the no-binary-always-install-via-wheel-sbi branch from 42d3f75 to 6e66f48 Compare April 5, 2021 14:09
news/9778.feature.rst Outdated Show resolved Hide resolved
def __init__(self, cache_dir, format_control=None):
# type: (str, Optional[FormatControl]) -> None
if format_control is None:
format_control = FormatControl()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not explicitly pass a fresh FormatControl() from the caller instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultimately the wheel cache will not be concerned about format control at all, so I wanted to already expose a constructor without FormatControl.

Copy link
Member

@pradyunsg pradyunsg Apr 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our cache logic is very much tied to the FormatControl class, and I'd prefer to make this not make this optional. Having this be an explicit dependency was an intentional choice, to make it clearer what the object needs to function.

(we have enough implicit dependencies in the rest of the codebase. :P)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Further thinking about this I'll see if I can make this PR not touch the caching use logic. Contrarily to what I initially thought it might be possible, and we can revisit the cache logic independently (which is the topic of #9162).

def check_binary_allowed(req):
# type: (InstallRequirement) -> bool
if "always-install-via-wheel" in features_enabled:
return True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be cleaner to have this logic outside of this function, in run() instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, because the point is to get a BinaryAllowedPredicate that is passed around down to _should_build.

@sbidoul sbidoul marked this pull request as draft April 5, 2021 21:11
@sbidoul sbidoul force-pushed the no-binary-always-install-via-wheel-sbi branch 4 times, most recently from d8c4571 to 3a92068 Compare April 6, 2021 17:36
@sbidoul sbidoul marked this pull request as ready for review April 6, 2021 17:37
@sbidoul
Copy link
Member Author

sbidoul commented Apr 6, 2021

This should now be good for review again. I removed the part that touched wheel caching, as it turns out it's an orthogonal topic.

@sbidoul
Copy link
Member Author

sbidoul commented Apr 7, 2021

The RTD failure seems unrelated ?

@pradyunsg
Copy link
Member

Yes, because the towncrier-drafts plugin is broken now.

@BrownTruck
Copy link
Contributor

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will be eligible for code review and hopefully merging!

@BrownTruck BrownTruck added the needs rebase or merge PR has conflicts with current master label Apr 7, 2021
In itself, this commit does not change pip's behaviour,
because the presence of --build-option and --global-option imply
--no-binary :all: which in turn disable wheel building.
--no-binary does not disable wheel building anymore, so a wheel
is built during the install process and setup.py install is
not used anymore in this case

--build-option and --global-option are passed to setup.py bdist_wheel
in the pip install command, like it is done in the pip wheel command.
@sbidoul sbidoul force-pushed the no-binary-always-install-via-wheel-sbi branch from 3a92068 to 5588997 Compare April 8, 2021 14:23
@pypa-bot pypa-bot removed the needs rebase or merge PR has conflicts with current master label Apr 8, 2021
@uranusjr
Copy link
Member

@sbidoul Do you want this in 21.2?

@uranusjr uranusjr added this to the 21.2 milestone Jul 12, 2021
@sbidoul
Copy link
Member Author

sbidoul commented Jul 12, 2021

I will not be ready in time I'm afraid. If I can scrape some time I'll rather focus on landing PEP 660.
Regarding this, I plan to continue it: I need to think again about the suggestions in #9769 (comment) and the interaction with the wheel cache.

@sbidoul sbidoul modified the milestones: 21.2, 21.3 Jul 12, 2021
@github-actions github-actions bot added the needs rebase or merge PR has conflicts with current master label Aug 15, 2021
@sbidoul sbidoul removed this from the 21.3 milestone Aug 29, 2021
@sbidoul sbidoul marked this pull request as draft August 9, 2022 14:45
@sbidoul sbidoul mentioned this pull request Aug 12, 2022
5 tasks
@sbidoul
Copy link
Member Author

sbidoul commented Aug 12, 2022

This continues in #11373.

@sbidoul sbidoul closed this Aug 12, 2022
@sbidoul sbidoul deleted the no-binary-always-install-via-wheel-sbi branch August 12, 2022 16:28
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
needs rebase or merge PR has conflicts with current master
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants