Keyring prevent some package uninstall #8443
Comments
triage-new-issues
bot
added
the
S: needs triage
|
@pradyunsg what do I need to do to get this triaged? |
|
May I also suggest filing an issue with keyring on this? I also found some closed issues related to cffi and Also I couldn't reproduce this issue in OSX Catalina 10.15.4 |
triage-new-issues
bot
removed
the
S: needs triage
|
Installers like pip have very strict constraints on what they can do. One of the reasons we vendor all of our dependencies is so that we don't typically hit issues like this. Keyring is an exception, being an "optional" dependency, but in this case the issue is that it doesn't conform to the restrictions that "being part of pip" impose (specifically, it depends on an external C extension. I don't think we can expect keyring to work under the limitations needed to make it safe for pip to use, unfortunately. One thing that might improve this issue is for pip to lazily import keyring only when needed. This would at least avoid the problem when keyring isn't actually used by pip (although there will still be potential issues - users who need keyring to access an index won't be able to upgrade cfffi without disabling keyring, for example). The issue is Windows-only because Windows won't allow you to delete a shared library that's loaded in a running process. Unix will allow you to do this, and so uninstalling cffi while using it is OK on Unix but not on Windows. |
|
@deveshks, the issue is not in keyring it's in pip and was introduced by #5952. Uninstalling a package using pip should not make use of any external (not in pip vendor). This is causing the issue that we have here. |
|
Thank you @pfmoore, yes, lazily importing keyring only when needed will fix the uninstall issue. But potentially still cause an issue on updates. However, for the update case, it will be probably be easier to diagnose (for a user that uses keyring to store pip related credentials) versus a user that have no idea keyring is installed (potentially via a dependency) and hits this issues... |
It's possible that some additional thought is needed around how keyring support is activated. I didn't get involved in the discussions at the time so I can't say whether a manual opt-in (beyond simply "having keyring installed") was considered and/or found unacceptable. |
|
Can any of the Member look at this issue and provide a fix? |
|
Please understand that keyring support is optional and the reason it's not built in is precisely because it can cause issues like this. There's no "fix" needed here. If you want to uninstall a package that keyring relies on, you may have to disable or remove keyring while you do so. Repeatedly insisting that someone else fix this issue isn't going to alter that fact. I've offered some suggestions as to how someone could try to reduce the number of times this is an issue, but I'm certainly not offering to do that work. And on reflection, I no longer consider the current behaviour a pip bug. As has been already suggested, another approach you could take is to raise an issue with keyring, saying that its use of C extensions causes this problem when it gets used in pip. But I doubt the keyring maintainers are likely to remove all C dependencies just for this very niche problem. I'm going to mark this issue as a feature request, and "Deferred till PR", to reflect the fact that there doesn't appear to be much more to say here until someone (quite likely not one of the pip committers, but a user who is sufficiently interested in improving the behaviour in this situation to work on it) submits a PR that can act as a starting point for further discussion. I'm happy if any of the other @pypa/pip-committers want to pick this up and handle it differently, but I will leave the discussion at this point. |
pfmoore
added
resolution: deferred till PR
|
@pfmoore I fully agree that However, currently because I am sure there are good reasons why The issue is not in This is not how a optional feature should work. |
|
The latest release of keyring 21.6.0 is less aggressive about initializing the keyring backend on import. It's possible this change will fix this reported issue. Please test and let me know. |
|
Looking at the code in pywin32-ctypes, it seems that it may be possible for pip to force the fallback to ctypes with something like: |
I've been diagnosing this for hours and just found this post... |
|
Just an update that tried 21.6.0, and 22.0.1 versions of keyring. Issue still there, and reproducible. |
I was hoping that by late-binding the lookup of backends that pip's use of keyring would not come into play for these use-cases, but it seems that's not the case. I tried to find out where keyring behavior is being triggered, but wasn't able to trigger it with an install/uninstall operation: pip bugfix/8443-block-cffi $ git diff
diff --git a/src/pip/_internal/network/auth.py b/src/pip/_internal/network/auth.py
index bd54a5cba..d6565e38d 100644
--- a/src/pip/_internal/network/auth.py
+++ b/src/pip/_internal/network/auth.py
@@ -37,6 +37,7 @@ except Exception as exc:
def get_keyring_auth(url, username):
+ breakpoint()
# type: (Optional[str], Optional[str]) -> Optional[AuthInfo]
"""Return the tuple auth for a given url from keyring."""
global keyringAs you can see, I added a breakpoint at auth.py:40 and then with that copy of pip and with keyring installed, installed CFFI and then uninstalled it, all without triggering the keyring behavior. I need to do more testing on Windows. |
|
So, using that technique on Windows, I'm able to replicate the issue with And the issue goes away with keyring 21.6: So there seems to be another factor at play if you're still encountering the issue after keyring 21.6. Is there something about your environment that's causing the 'auth' behavior of pip to be invoked? |
|
pip 20.3.1, keyring 21.6.0, cffi 1.13.2 |
|
@jaraco, try this: |
|
I couldn't install cffi 1.13.2 due to compiler configuration, but I was able to replicate the issue thus: |
|
Looks like the issue was fixed in #8687. |
Environment
Description
Optional pip keyring support is cause some resource lock while uninstalling cffi
Expected behavior
pip should not lock any resources and allow clean uninstall of cffi
How to Reproduce
Output
To show that pip keyring feature is in cause here, setting the keyring backend to null (set PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring) is preventing this issue.
The text was updated successfully, but these errors were encountered: