Docker build fails on installing pydantic-core [linux/arm/v7]

[return42]

Docker build fails on installing pydantic-core / see report from CI:

• https:/searxng/searxng/actions/runs/11191050608/job/31113678686#step:8:981

#24 [linux/arm/v7 5/8] RUN apk add --no-cache -t build-dependencies     build-base     py3-setuptools     python3-dev     libffi-dev     libxslt-dev     libxml2-dev     openssl-dev     tar     git  && apk add --no-cache     ca-certificates     python3     py3-pip     libxml2     libxslt     openssl     tini     uwsgi     uwsgi-python3     brotli  && pip3 install --break-system-packages --no-cache -r requirements.txt  && apk del build-dependencies  && rm -rf /root/.cache
...
#24 221.9 Collecting pydantic-core==2.20.1 (from pydantic==2.8.2->-r requirements.txt (line 19))
#24 221.9   Downloading pydantic_core-2.20.1.tar.gz (388 kB)
#24 222.0      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 388.4/388.4 kB 18.9 MB/s eta 0:00:00
#24 222.3   Installing build dependencies: started
#24 230.0   Installing build dependencies: finished with status 'done'
#24 230.0   Getting requirements to build wheel: started
#24 230.6   Getting requirements to build wheel: finished with status 'done'
#24 230.6   Preparing metadata (pyproject.toml): started
#24 231.2   Preparing metadata (pyproject.toml): finished with status 'error'
#24 231.2   error: subprocess-exited-with-error
#24 231.2   
#24 231.2   × Preparing metadata (pyproject.toml) did not run successfully.
#24 231.2   │ exit code: 1
#24 231.2   ╰─> [6 lines of output]
#24 231.2       
#24 231.2       Cargo, the Rust package manager, is not installed or is not on PATH.
#24 231.2       This package requires Rust and Cargo to compile extensions. Install it through
#24 231.2       the system's package manager or via https://rustup.rs/
#24 231.2       
#24 231.2       Checking for Rust toolchain....
#24 231.2       [end of output]
#24 231.2   
#24 231.2   note: This error originates from a subprocess, and is likely not a problem with pip.
#24 231.2 error: metadata-generation-failed
#24 231.2 
#24 231.2 × Encountered error while generating package metadata.
#24 231.2 ╰─> See above for output.
#24 231.2 
#24 231.2 note: This is an issue with the package mentioned above, not pip.
#24 231.2 hint: See above for details.
#24 ERROR: process "/bin/sh -c apk add --no-cache -t build-dependencies     build-base     py3-setuptools     python3-dev     libffi-dev     libxslt-dev     libxml2-dev     openssl-dev     tar     git  && apk add --no-cache     ca-certificates     python3     py3-pip     libxml2     libxslt     openssl     tini     uwsgi     uwsgi-python3     brotli  && pip3 install --break-system-packages --no-cache -r requirements.txt  && apk del build-dependencies  && rm -rf /root/.cache" did not complete successfully: exit code: 1
------
 > [linux/arm/v7 5/8] RUN apk add --no-cache -t build-dependencies     build-base     py3-setuptools     python3-dev     libffi-dev     libxslt-dev     libxml2-dev     openssl-dev     tar     git  && apk add --no-cache     ca-certificates     python3     py3-pip     libxml2     libxslt     openssl     tini     uwsgi     uwsgi-python3     brotli  && pip3 install --break-system-packages --no-cache -r requirements.txt  && apk del build-dependencies  && rm -rf /root/.cache:
231.2       [end of output]
231.2   
231.2   note: This error originates from a subprocess, and is likely not a problem with pip.
231.2 error: metadata-generation-failed
231.2 
231.2 × Encountered error while generating package metadata.
231.2 ╰─> See above for output.
231.2 
231.2 note: This is an issue with the package mentioned above, not pip.
231.2 hint: See above for details.
------

 1 warning found (use docker --debug to expand):
 - SecretsUsedInArgOrEnv: Do not use ARG or ENV instructions for sensitive data (ENV "MORTY_KEY") (line 12)
Dockerfile:26
--------------------
  25 |     
  26 | >>> RUN apk add --no-cache -t build-dependencies \
  27 | >>>     build-base \
  28 | >>>     py3-setuptools \
  29 | >>>     python3-dev \
  30 | >>>     libffi-dev \
  31 | >>>     libxslt-dev \
  32 | >>>     libxml2-dev \
  33 | >>>     openssl-dev \
  34 | >>>     tar \
  35 | >>>     git \
  36 | >>>  && apk add --no-cache \
  37 | >>>     ca-certificates \
  38 | >>>     python3 \
  39 | >>>     py3-pip \
  40 | >>>     libxml2 \
  41 | >>>     libxslt \
  42 | >>>     openssl \
  43 | >>>     tini \
  44 | >>>     uwsgi \
  45 | >>>     uwsgi-python3 \
  46 | >>>     brotli \
  47 | >>>  && pip3 install --break-system-packages --no-cache -r requirements.txt \
  48 | >>>  && apk del build-dependencies \
  49 | >>>  && rm -rf /root/.cache
  50 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c apk add --no-cache -t build-dependencies     build-base     py3-setuptools     python3-dev     libffi-dev     libxslt-dev     libxml2-dev     openssl-dev     tar     git  && apk add --no-cache     ca-certificates     python3     py3-pip     libxml2     libxslt     openssl     tini     uwsgi     uwsgi-python3     brotli  && pip3 install --break-system-packages --no-cache -r requirements.txt  && apk del build-dependencies  && rm -rf /root/.cache" did not complete successfully: exit code: 1
ERROR: docker.build exit with error (1)
make: *** [Makefile:94: docker.buildx] Error 1
Error: Process completed with exit code 2.

---

@vojkovic pointed out wheel pydantic_core-2.20.1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl at pydantic-core/2.20.1/#files

[return42]

If I'm not wrong alpine linux is musl and there does not exists a pre-build musl for pydantic-core .. may it helps when we install it by the alpine package manager?

• https://pkgs.alpinelinux.org/package/edge/community/armv7/py3-pydantic-core

[unixfox]

Hi @vojkovic, Can I let you handle the fix since your PR broke the docker CI?

Thanks

[return42]

PR #3889 (comment) did not solve the issue / I reopen this bug report.

[vojkovic]

See: #3889 (comment)

[unixfox]

Cargo, the Rust package manager, is not installed or is not on PATH.
This package requires Rust and Cargo to compile extensions. Install it through

Can't we install Cargo in the Dockerfile in order to solve the issue?

[return42]

FYI: Bugfix is on the way:

• ci: add prebuilt musl for armv7 pydantic/pydantic-core#1477

[unixfox]

FYI: Bugfix is on the way:

* [ci: add prebuilt musl for armv7 pydantic/pydantic-core#1477](https:/pydantic/pydantic-core/pull/1477)

What an awesome timing!

[vojkovic]

@return42 pydantic/pydantic-core#1477

Merged!

[return42]

Yeah :-) .. merged! .. but we still have to wait for a new release of

• pydantic-core and
• pydantic

[vojkovic]

we still have to wait for a new release

Ahh, shame - looks like the last release was last month, so I hope we can see a new release from them soon. What's your opinion on temporarily disabling armv7 builds until the new release comes?

[unixfox]

What about doing what I explained in #3887 (comment) then use the prebuilt pip release when it is available.

[return42]

What about doing what I explained in #3887 (comment) then use the prebuilt pip release when it is available.

Before we investigate to much, we should wait, the next pydantic version is on the way --> pydantic/pydantic#10605 (comment)

[unixfox]

Then we should revert #3727 until the new pydantic version is out. No new docker images are being built, that's a big issue!

[return42]

Then we should revert #3727 until the new pydantic version is out.

Personally I don't want to merge quirks or destroy the history of the SearXNG core when it is about a deployment problem on a rare used architecture ..

No new docker images are being built, that's a big issue!

Yeah, I fully agree .. but I also still waiting about a PR from the docker user group .. if none PR comes, we have to wait the new pydantic version is released at some day .. I see the community of docker users as responsible for dealing with this problem / the problem is not in SearXNG core.

[unixfox]

Yeah, I fully agree .. but I also still waiting about a PR from the docker user group .. if none PR comes, we have to wait the new pydantic version is released at some day .. I see the community of docker users as responsible for dealing with this problem / the problem is not in SearXNG core.

Docker is the most used way to install SearXNG. We should certainly not block the entire Docker build due to a new feature. All Docker builds are broken in all CPU architectures because Docker build expect all the CPU architectures to be built successfully in order to create a new Docker image.

Obviously this is a big issue and should be handled like if it was part of the core. If 80% of our users can't update their installation, how are they going to get the new code changes??? This doesn't make any sense... Thinking the contrary is like thinking "it works on my machine, I don't care about the others".

I don't have time to invest into any long fixes, I can recommend things, but I'm busy on other open source projects (and personal ones). Hence, why I haven't really pushed anyway for fixing this.

IMO if something broke an essential part of SearXNG, we shouldn't keep it for too long. It has been 7 days with a broken Docker build. It's worse than having reverted commits into the git log...

---

Then we should revert #3727 until the new pydantic version is out.

Personally I don't want to merge quirks or destroy the history of the SearXNG core when it is about a deployment problem on a rare used architecture ..

I can say the same for LXC support! Like who cares about a software that has a direct and better replacement: Docker!

Jokes aside... If you start to support something, then you continue to support it until it is really possible. You adapt your code so that it's still compatible. The only time you can drop the support for something is when a major feature won't work on this device or CPU architecture or software AND you can't do anything about it, there is no choice.

And dropping the support for something should be made with an announcement, not silently because a new code change doesn't work with it.

[return42]

I don't have time to invest into any long fixes, ..

Sorry I can't help .. I don't have the docker nor the GH CI action knowledge for .. but I still awaiting PRs.

And dropping the support for something should be made with an announcement,

Dropping architectures was never an option of this issue.

I can say the same for LXC support! Like who cares about a software that has a direct and better replacement: Docker!

The plan is to remove the LXC part from SearXNG in the near future.

[rhee876527]

@return42 The Dockerfile can cater to all the architectures if you conditionally ignore pydantic in the requirements.txt for the arm build (32-bit) like this.

ARG TARGETARCH
RUN if [ "$TARGETARCH" = "arm" ]; then \
        apk add --no-cache py3-pydantic && pip install --no-cache --break-system-packages -r <(grep -v '^pydantic' requirements.txt); \
    else \
        pip install --no-cache --break-system-packages -r requirements.txt; \
    fi

This should allow the docker builds to resume in the meantime as you await an unknown release date from pydantic. Would you want a PR for it?

[return42]

This should allow the docker builds to resume in the meantime as you await an unknown release date from pydantic.

@rhee876527 yes, why not .. everything that helps is welcome :)

However, I have to say that I don't have any know-how about the docker (& -hub) ecosystem , I can't estimate what effects this has for docker users and therefore I can't help if there are questions in the review ..

[return42]

I reopen this issue since #3936 is just an interim solution.

[return42]

FYI new docker builds are available: https://hub.docker.com/r/searxng/searxng/tags

I think the final solution to this issue will be the move from pydantic to msgspec / see

• [refactor] replace pydantic by msgspec #3938

testers are welcome ;-)