-
Notifications
You must be signed in to change notification settings - Fork 150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python 3.11 MacOS x86: Wheels are always marked as universal2
#573
Comments
By default, it picks up whatever CPython is built with. If you use a universal2 copy of Python, it guesses you are making universal wheels too. If you are making redistrubtable wheels, I highly recommend cibuildwheel, which does all of the settings for you, including making sure it uses the official CPython releases, as CIs often don't provide the backward compat (10.9) that the official downloads do. But you can work out all this by hand if you really want to. I don't remember if this one is an envvar, or if you just give |
Alright, with cibuildwheel and explicitly setting the architecture this works. The only pain is the inconsistent naming of architectures between Windows and MacOS, but that's another issue... But in general, is this behavior documented somewhere? Or what it means to "post-process" a wheel? I can't find anything on that on the packaging guides. |
I have the same issue exactly in my current project and I've written about it extensively in one of my project's devnotes here. I'll post the last part which explores possible solutions: Tried the following:
What to do? |
You are building with a universal2 build of Python. Setuptools will ask Python what it was built as and uses that. You can set |
Thanks very much for your help on this. I will check out the cibuildwheel code! |
Thanks again, @henryiii In case anyone else faces the same issues I did, here's some illustrative code which resolves the issue as per your advice: #!/usr/bin/env python3
import os
import platform
import sys
def build_wheel(universal=False):
"""wheel build config (special cases macos wheels) for github runners
ref: https:/pypa/cibuildwheel/blob/main/cibuildwheel/macos.py
"""
assert sys.version_info.minor >= 8, "applies to python >= 3.8"
_platform = platform.system()
_arch = platform.machine()
_cmd = "python3 setup.py bdist_wheel"
if _platform == "Darwin":
_min_osx_ver = "10.9"
if _arch == 'arm64' and not universal:
_min_osx_ver = "11.0"
if universal:
_prefix = (f"ARCHFLAGS='-arch arm64 -arch x86_64' "
f"_PYTHON_HOST_PLATFORM='macosx-{_min_osx_ver}-universal2'")
else:
_prefix = (f"ARCHFLAGS='-arch {_arch}' "
f"_PYTHON_HOST_PLATFORM='macosx-{_min_osx_ver}-{_arch}'")
_cmd = " ".join([_prefix, _cmd])
os.system(_cmd)
if __name__ == '__main__':
build_wheel() |
In my case, another workaround I found was to manually override the platform tag., i.e., this code snippet from setuptools import setup
from wheel.bdist_wheel import bdist_wheel, get_platform
class CustomWheel(bdist_wheel):
"""Override platform tags when building a wheel."""
def initialize_options(self):
super().initialize_options()
def finalize_options(self):
platform_name = get_platform("_") # any `str` object works
if ("universal2" in platform_name):
self.plat_name = platform_name.replace("universal2", "x86_64") # or your host/target architecture
def run(self):
super.run()
setup(cmdclass={"bdist_wheel": CustomWheel}, *args, **kwargs) works for me, though in hindsight I should have used the tuple that is returned from I did end up switching to |
Thanks for the alternative solution, @agriyakhetarpal I actually tried the Ideally, in the case that a |
I'd highly recommend using cibuildwheel. There are several other concerns; for example, you don't want to use the GitHub Actions compiled Python or brew's compiled Python, as those don't target macOS 10.9, like the official binaries do, so you should always download the official binaries and use those when building redistributable wheels. Cross-compiling to Windows ARM is even more complex, and targeting manylinux/musllinux requires a different workflow. cibuildwheel solves all of these issues, and can be used locally too if you want. It also works even if you are not using wheel, such as when using maturin (Rust), scikit-build-core, or meson-python, none of which can use wheel since there's no public API (and maturin is also itself written in Rust). Flags should not be added to the I'd also avoid customizing |
Thanks for the advice, @henryiii
In my particular case, for the current project I'm working on, the build process is a little bit involved already with a Everything was working fine on local machines, until I decided to add github actions to the equation to help build across a number of platforms {macos, linux} x {arm64, x86_64} and then I had the issues which you previously helped me to resolve. During the latter part of the above process, I actually looked at
Understood.
Fair enough. |
Many thanks for your comments @henryiii. If you have a project that is still using
|
I would recommend all projects have a Cibuildwheel is a frontend; it doesn't care if you use |
I have two seperate projects that use two different approaches to build Platform Wheels, and both generate an
universal2
wheel on MacOS 3.11 with Python x86_64 that in reality is NOT an universal2 wheel, but rather x86 only. I think there is something wrong with the wheel platform detection.python setup.py bdist_wheel
approach and is a setuptools+setuptools_rust project. Example GitHub Actions run that showsuniversal2
being generated: https:/SkyTemple/skytemple-rust/actions/runs/6355353799/job/17263321360pip wheel
and uses setuptools+setuptools_dso: https:/SkyTemple/tilequant/actions/runs/6225727358/job/16896773186This does not happen with 3.8 - 3.11.
I am only able to reproduce this with GitHub Actions, since I don't have a Mac. I could imagine this being an issue in GitHub Actions, but I find that to be unlikely. I haven't really found any other project that publishes these kind of wheels and doesn't use GitHub Actions, so I was not able to confirm this.
The text was updated successfully, but these errors were encountered: