Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PIP_EXTRA_INDEX_URL not used. Multiple artifacts feeds not parsed correctly. #1688

Closed
MarcSkovMadsen opened this issue Feb 19, 2024 · 7 comments
Labels
duplicate This issue or pull request already exists

Comments

@MarcSkovMadsen
Copy link

MarcSkovMadsen commented Feb 19, 2024

My team uses a PIP_EXTRA_INDEX_URL environment variable to point to multiple azure artifacts feeds.

If I try to use uv

  • the PIP_EXTRA_INDEX_URL variable seems not to be used and
  • --extra-index-url does not work with a space separated list of artifact feeds.

Reproduce Basic Issue

  • export PIP_EXTRA_INDEX_URL=https://USER:[email protected]/ORGANISATION/_packaging/FEED%40Local/pypi/simple/
  • uv pip install CUSTOM_PACKAGE
  × No solution found when resolving dependencies:
  ╰─▶ Because CUSTOM_PACKAGE was not found in the package registry and you require CUSTOM_PACKAGE, we can conclude that the requirements are unsatisfiable.

If I run

  • pip install CUSTOM_PACKAGE I get it installed

Issue with --extra-index-url

If I try

uv pip install CUSTOM_PACKAGE --extra-index-url=$PIP_EXTRA_INDEX_URL

I get

error: Failed to download: CUSTOM_PACKAGE==0.1.1110624
  Caused by: HTTP status client error (405 Method Not Allowed) for url (https://pkgs.dev.azure.com/ORGANISATION/_packaging/SOME_ID@ANOTHER_ID/pypi/download/CUSTOM_PACKAGE/0.1.1110624/CUSTOM_PACKAGE-0.1.1110624-py3-none-any.whl#sha256=SOME_SHA256_CODE)

This is the same issue as #1371.

Issue with multiple artifacts feeds

In practice we use multiple azure artifacts feeds. We set PIP_EXTRA_INDEX_URL to a space separated list of the urls.

export PIP_EXTRA_INDEX_URL="https://USER:[email protected]/ORGANISATION/_packaging/FEED_1%40Local/pypi/simple/ https://USER:[email protected]/ORGANISATION/_packaging/FEED_2/pypi/simple/"

The PIP_EXTRA_INDEX_URL is again not taken into account

uv pip install CUSTOM_PACKAGE
  × No solution found when resolving dependencies:
  ╰─▶ Because CUSTOM_PACKAGE was not found in the package registry and you require CUSTOM_PACKAGE, we can conclude that the requirements are unsatisfiable.

If I try to work around it using --extra-index-url i see

 uv pip install CUSTOM_PACKAGE --extra-index-url="$PIP_EXTRA_INDEX_URL"
error: HTTP status client error (400 Bad Request) for url (https://pkgs.dev.azure.com/ORGANISATION/_packaging/FEED_1%40Local/pypi/simple/%20https://USER:[email protected]/ORGANISATION/_packaging/FEED_2/pypi/simple/CUSTOM_PACKAGE/)

Thus uv does not handle multiple feeds correctly.

Using pip works

pip install CUSTOM_PACKAGE --extra-index-url="$PIP_EXTRA_INDEX_URL"
...
Successfully installed CUSTOM_PACKAGE-0.1.1110624.
@notatallshaw
Copy link
Contributor

notatallshaw commented Feb 19, 2024

FYI, for any use case other than a mirror, pip extra index url is inheriently insecure, as pip does not guarantee ordering of which index url it will read first an attacker can place a package on the other index with the same name as your internal package. This isn't theoretical, malicious packages were uploaded to PyPI that mirrored the names of packages on the pytorch index.

The attempt to solve this is PEP 708, but it hasn't been implemented by PyPI yet. But I would strongly advise no one use extra index url with pip unless they very specifically are hosting multiple index mirrors.

For uv see related discussion here: #171

@charliermarsh
Copy link
Member

Thanks! The env var is a duplicate of #1535. The 401 is a duplicate of #1458.

@charliermarsh charliermarsh closed this as not planned Won't fix, can't repro, duplicate, stale Feb 19, 2024
@notatallshaw
Copy link
Contributor

The env var is a duplicate of #1535.

FYI pip index url and pip extra index url are two different features of Pip. The latter allowing to query multiple indexes (and as I've said is inherently insecure).

You seem to have linked to the former and this request is about the latter.

@charliermarsh
Copy link
Member

@notatallshaw - Ah sorry, I just consider it "the same" underlying issue, since if we add support for PIP_INDEX_URL via an environment variable, we'll naturally do the same for PIP_EXTRA_INDEX_URL.

@zanieb zanieb added the duplicate This issue or pull request already exists label Feb 19, 2024
@MarcSkovMadsen
Copy link
Author

Thanks @charliermarsh . I don't think the two mentioned existing issues covers the "Issue with multiple artifacts feeds". Would you consider and reopen if you agree. Thanks.

@zanieb
Copy link
Member

zanieb commented Feb 19, 2024

I can open an issue to track that — although I think the solution here is to pass --extra-index-url multiple times.

@zanieb
Copy link
Member

zanieb commented Feb 19, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

4 participants