Can't install with pip in alpine #31

yuukh · 2023-03-24T12:42:07Z

the latest version ( 0.4.0 ) can't be installed

My environment:
OS: arm64 docker alpine
Python: 3.8.10
Pip: 23.0.1

here is the error log

  Using cached curl_cffi-0.4.0.tar.gz (75 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [37 lines of output]
      Traceback (most recent call last):
        File "/usr/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 16, in <module>
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 108, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 147, in setup
          _setup_distribution = dist = klass(attrs)
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 488, in __init__
          _Distribution.__init__(
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 283, in __init__
          self.finalize_options()
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 912, in finalize_options
          ep(self)
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 932, in _finalize_setup_keywords
          ep.load()(self, ep.name, value)
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/cffi/setuptools_ext.py", line 219, in cffi_modules
          add_cffi_module(dist, cffi_module)
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module
          execfile(build_file_name, mod_vars)
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/cffi/setuptools_ext.py", line 25, in execfile
          exec(code, glob, glob)
        File "curl_cffi/build.py", line 6, in <module>
          ffibuilder = FFI()
        File "/tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/cffi/api.py", line 48, in __init__
          import _cffi_backend as backend
      ImportError: Error loading shared library /tmp/pip-build-env-5aj63q6z/overlay/lib/python3.8/site-packages/_cffi_backend.cpython-38-aarch64-linux-gnu.so: Operation not permitted
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.```

The text was updated successfully, but these errors were encountered:

perklet · 2023-03-24T12:48:00Z

Sorry, there is no precompiled wheel for alpine(musl-libc) based Linux for now. You will have to build from source, including upstream projects.

yuukh · 2023-03-24T12:51:23Z

可以给一点具体的指导吗

yuukh · 2023-03-24T13:22:40Z

/ # pip wheel curl_cffi
Collecting curl_cffi
  Using cached curl_cffi-0.4.0.tar.gz (75 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting cffi>=1.12.0
  Using cached cffi-1.15.1-cp38-cp38-linux_aarch64.whl
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Saved /cffi-1.15.1-cp38-cp38-linux_aarch64.whl
Saved /pycparser-2.21-py2.py3-none-any.whl
Building wheels for collected packages: curl_cffi
  Building wheel for curl_cffi (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for curl_cffi (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [114 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-aarch64-cpython-38
      creating build/lib.linux-aarch64-cpython-38/curl_cffi
      copying curl_cffi/_const.py -> build/lib.linux-aarch64-cpython-38/curl_cffi
      copying curl_cffi/__init__.py -> build/lib.linux-aarch64-cpython-38/curl_cffi
      copying curl_cffi/build.py -> build/lib.linux-aarch64-cpython-38/curl_cffi
      creating build/lib.linux-aarch64-cpython-38/curl_cffi/requests
      copying curl_cffi/requests/cookies.py -> build/lib.linux-aarch64-cpython-38/curl_cffi/requests
      copying curl_cffi/requests/session.py -> build/lib.linux-aarch64-cpython-38/curl_cffi/requests
      copying curl_cffi/requests/__init__.py -> build/lib.linux-aarch64-cpython-38/curl_cffi/requests
      copying curl_cffi/requests/headers.py -> build/lib.linux-aarch64-cpython-38/curl_cffi/requests
      copying curl_cffi/requests/errors.py -> build/lib.linux-aarch64-cpython-38/curl_cffi/requests
      running egg_info
      writing curl_cffi.egg-info/PKG-INFO
      writing dependency_links to curl_cffi.egg-info/dependency_links.txt
      writing requirements to curl_cffi.egg-info/requires.txt
      writing top-level names to curl_cffi.egg-info/top_level.txt
      reading manifest file 'curl_cffi.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      /tmp/pip-build-env-1j40w2gb/overlay/lib/python3.8/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
        warnings.warn(msg, _BetaConfiguration)
      warning: no files found matching 'curl_cffi/cacert.pem'
      warning: no files found matching 'curl_cffi/*.dll'
      adding license file 'LICENSE'
      writing manifest file 'curl_cffi.egg-info/SOURCES.txt'
      copying curl_cffi/cdef.c -> build/lib.linux-aarch64-cpython-38/curl_cffi
      copying curl_cffi/shim.c -> build/lib.linux-aarch64-cpython-38/curl_cffi
      creating build/lib.linux-aarch64-cpython-38/curl_cffi/include
      copying curl_cffi/include/shim.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include
      creating build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/Makefile.am -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/curl.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/curlver.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/easy.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/header.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/mprintf.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/multi.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/options.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/stdcheaders.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/system.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/typecheck-gcc.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      copying curl_cffi/include/curl/urlapi.h -> build/lib.linux-aarch64-cpython-38/curl_cffi/include/curl
      running build_ext
      generating cffi module 'build/temp.linux-aarch64-cpython-38/curl_cffi._wrapper.c'
      creating build/temp.linux-aarch64-cpython-38
      building 'curl_cffi._wrapper' extension
      creating build/temp.linux-aarch64-cpython-38/build
      creating build/temp.linux-aarch64-cpython-38/build/temp.linux-aarch64-cpython-38
      creating build/temp.linux-aarch64-cpython-38/curl_cffi
      gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Icurl_cffi/include -I/usr/local/include/python3.8 -c build/temp.linux-aarch64-cpython-38/curl_cffi._wrapper.c -o build/temp.linux-aarch64-cpython-38/build/temp.linux-aarch64-cpython-38/curl_cffi._wrapper.o
      gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Icurl_cffi/include -I/usr/local/include/python3.8 -c curl_cffi/shim.c -o build/temp.linux-aarch64-cpython-38/curl_cffi/shim.o
      curl_cffi/shim.c: In function '_curl_easy_setopt':
      curl_cffi/shim.c:9:16: warning: unused variable 'opt_value' [-Wunused-variable]
          9 |     CURLoption opt_value = (CURLoption) option;
            |                ^~~~~~~~~
      gcc -shared build/temp.linux-aarch64-cpython-38/build/temp.linux-aarch64-cpython-38/curl_cffi._wrapper.o build/temp.linux-aarch64-cpython-38/curl_cffi/shim.o -L/usr/local/lib -L/usr/local/lib -lcurl-impersonate-chrome -o build/lib.linux-aarch64-cpython-38/curl_cffi/_wrapper.abi3.so
      /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: cannot find -lcurl-impersonate-chrome: No such file or directory
      collect2: error: ld returned 1 exit status
      /tmp/pip-build-env-1j40w2gb/overlay/lib/python3.8/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning:     Installing 'curl_cffi.include' as data is deprecated, please list it in `packages`.
          !!
      
      
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'curl_cffi.include' as an importable package,
          but it is not listed in the `packages` configuration of setuptools.
      
          'curl_cffi.include' has been automatically added to the distribution only
          because it may contain data files, but this behavior is likely to change
          in future versions of setuptools (and therefore is considered deprecated).
      
          Please make sure that 'curl_cffi.include' is included as a package by using
          the `packages` configuration field or the proper discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
      
          You can read more about "package discovery" and "data files" on setuptools
          documentation page.
      
      
      !!
      
        check.warn(importable)
      /tmp/pip-build-env-1j40w2gb/overlay/lib/python3.8/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning:     Installing 'curl_cffi.include.curl' as data is deprecated, please list it in `packages`.
          !!
      
      
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'curl_cffi.include.curl' as an importable package,
          but it is not listed in the `packages` configuration of setuptools.
      
          'curl_cffi.include.curl' has been automatically added to the distribution only
          because it may contain data files, but this behavior is likely to change
          in future versions of setuptools (and therefore is considered deprecated).
      
          Please make sure that 'curl_cffi.include.curl' is included as a package by using
          the `packages` configuration field or the proper discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
      
          You can read more about "package discovery" and "data files" on setuptools
          documentation page.
      
      
      !!
      
        check.warn(importable)
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for curl_cffi
Failed to build curl_cffi
ERROR: Failed to build one or more wheels
WARNING: You are using pip version 22.0.4; however, version 23.0.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.

这个报错不知道如何解决

perklet · 2023-03-24T13:27:57Z

一两句话能说明白我不早就给你编译好了么😂

主要是这个错误：

-lcurl-impersonate-chrome: No such file or directory

你得把上游的 curl-impersonate 在 alpine 下编译好，然后放到 LD_LIBRARY_PATH，一般是 /usr/local/lib 里边。

yuukh · 2023-03-24T13:47:11Z

/tmp # ls /usr/local/lib/
curl-impersonate-chrome                           curl_ff102
curl-impersonate-ff                               curl_ff109
curl-impersonate-v0.5.4.aarch64-linux-gnu.tar.gz  curl_ff91esr
curl_chrome100                                    curl_ff95
curl_chrome101                                    curl_ff98
curl_chrome104                                    curl_safari15_3
curl_chrome107                                    curl_safari15_5
curl_chrome110                                    libpython3.8.so
curl_chrome99                                     libpython3.8.so.1.0
curl_chrome99_android                             libpython3.so
curl_edge101                                      pkgconfig
curl_edge99                                       python3.8
curl_ff100

还是一样的报错，是命名不对吗

perklet · 2023-03-27T03:39:08Z

应该是有 /usr/local/lib/libcurl-impersonate-chrome.so 这个文件才行

yuukh · 2023-03-28T04:59:39Z

编译上游curl-impersonate报错了，暂时解决不了

perklet · 2023-03-28T13:18:07Z

编译上游curl-impersonate报错了，暂时解决不了

这个倒是好说，上游有 alpine 的 docker 镜像，你直接把 so 复制出来就行了……

不过 alpine 和 cffi 好像不对付，我试了几下总是编译不过去。

yuukh · 2023-03-29T04:40:57Z

curl-impersonate alpine 的 docker 镜像只有amd4，我需要arm64
cffi问题你试试apk add py3-cffi

perklet · 2023-03-29T05:37:04Z

libcurl-impersonate-v0.5.4.aarch64-linux-musl.zip
我 build 过了，你试试行不行

yuukh · 2023-03-29T13:49:08Z

镜像选择docker hub 里的python:3.8.16-alpine3.17，libcurl-impersonate放在/usr/local/lib/下，运行

apk add --update build-base python3-dev libffi-dev curl‑dev
pip3 wheel curl_cffi

可成功编译，编译结果：curl_cffi-0.5.0-cp37-abi3-linux_aarch64.whl.zip

在其他环境中安装，

pip3 install /path/to/curl_cffi-0.5.0-cp37-abi3-linux_aarch64.whl

安装完成后，我还执行了两部才能正常使用
1.libcurl-impersonate放在/usr/local/lib/下
2.将cacert.pem放入Python\Lib\site-packages\curl_cffi\

不知道有没有办法实现安装完就能使用，不再手动操作

perklet · 2023-03-30T05:40:12Z

pypi 上的 wheels 都是用 cibuildwheel 编译出来的，你可以看下 .github/ 下面的配置，理论上来说我把你这个加进去就可以自动编译了，但是我还不太清楚之前我弄的哪里出了问题，你有时间的话可以先看下 .github/ 里的内容，我在 feature/musllinux 分支上做了些尝试，但是 master 上还没试过

yuukh · 2023-03-30T13:32:31Z

能确保走了这一行吗https:/yifeikong/curl_cffi/blob/feature/musllinux/prebuild.py#L56
这里有一个print，但我在workflow日志里没看见输出

perklet · 2023-03-31T02:22:21Z

能确保走了这一行吗https:/yifeikong/curl_cffi/blob/feature/musllinux/prebuild.py#L56 这里有一个print，但我在workflow日志里没看见输出

有的，你看下这里：https:/yifeikong/curl_cffi/actions/runs/4542689763/jobs/8006427035#step:5:4282

yuukh · 2023-04-01T10:48:29Z

不知道原因

thanhtoan1196 · 2023-10-27T03:31:16Z

any updates?

Doublefire-Chen · 2023-12-07T11:28:01Z

same issue
I just tried to install this package with pip in my docker container(Qing Long dashboard), whose package manager is apk. I failed. I think this issue is the same with the issue in alpine Linux. The key point is how to install libcurl-impersonate-chrome on alpine.

perklet · 2023-12-07T13:11:46Z

You can extract the alpine .so files from curl-impersonate's docker images.

Doublefire-Chen · 2023-12-07T13:35:20Z

You can extract the alpine .so files from curl-impersonate's docker images.

Thank your kind advice though, I quit. As far as I concerned, why not run my python script in the host machine using crontab.

perklet · 2024-03-06T09:24:38Z

Available since 0.6.3b1.

perklet added the enhancement New feature or request label May 23, 2023

perklet self-assigned this May 23, 2023

perklet added this to the v0.6 milestone Dec 11, 2023

perklet linked a pull request Dec 11, 2023 that will close this issue

Feature/impersonate 6.0 #163

Merged

2 tasks

k3it mentioned this issue Dec 24, 2023

Update duckduckgo_search(fix #490) n3d1117/chatgpt-telegram-bot#501

Closed

perklet removed a link to a pull request Dec 31, 2023

Feature/impersonate 6.0 #163

Merged

2 tasks

bashonly mentioned this issue Feb 27, 2024

[build] Add default optional dependency group yt-dlp/yt-dlp#9295

Merged

6 tasks

perklet linked a pull request Mar 6, 2024 that will close this issue

Add support for Alpine(musllinux) #264

Merged

perklet closed this as completed in #264 Mar 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't install with pip in alpine #31

Can't install with pip in alpine #31

yuukh commented Mar 24, 2023

perklet commented Mar 24, 2023 •

edited

Loading

yuukh commented Mar 24, 2023

yuukh commented Mar 24, 2023

perklet commented Mar 24, 2023

yuukh commented Mar 24, 2023

perklet commented Mar 27, 2023

yuukh commented Mar 28, 2023

perklet commented Mar 28, 2023

yuukh commented Mar 29, 2023

perklet commented Mar 29, 2023

yuukh commented Mar 29, 2023

perklet commented Mar 30, 2023 •

edited

Loading

yuukh commented Mar 30, 2023

perklet commented Mar 31, 2023

yuukh commented Apr 1, 2023

thanhtoan1196 commented Oct 27, 2023

Doublefire-Chen commented Dec 7, 2023

perklet commented Dec 7, 2023

Doublefire-Chen commented Dec 7, 2023

perklet commented Mar 6, 2024

Can't install with pip in alpine #31

Can't install with pip in alpine #31

Comments

yuukh commented Mar 24, 2023

perklet commented Mar 24, 2023 • edited Loading

yuukh commented Mar 24, 2023

yuukh commented Mar 24, 2023

perklet commented Mar 24, 2023

yuukh commented Mar 24, 2023

perklet commented Mar 27, 2023

yuukh commented Mar 28, 2023

perklet commented Mar 28, 2023

yuukh commented Mar 29, 2023

perklet commented Mar 29, 2023

yuukh commented Mar 29, 2023

perklet commented Mar 30, 2023 • edited Loading

yuukh commented Mar 30, 2023

perklet commented Mar 31, 2023

yuukh commented Apr 1, 2023

thanhtoan1196 commented Oct 27, 2023

Doublefire-Chen commented Dec 7, 2023

perklet commented Dec 7, 2023

Doublefire-Chen commented Dec 7, 2023

perklet commented Mar 6, 2024

perklet commented Mar 24, 2023 •

edited

Loading

perklet commented Mar 30, 2023 •

edited

Loading