Fix package permissions

[winsonluk]

This PR addresses issues that arise when packages in site-packages are not owned by the same user running pip.

Reproduce:

$ sudo pip install package        # installs into site-packages/package-1.0.dist-info with owner=root
$ pip install --upgrade package   # fails since $USER doesnt have permission to rename package-1.0.dist-info...
                                  #  ...but still generates site-packages/~ackage-1.0.dist-info with owner=$USER
$ pip freeze                      # WARNING when parsing site-packages/~ackage-1.0.dist-info

Changes:

• Fail fast when upgrading packages: Currently, if a user tries to upgrade a package that they don't own, a temporary folder is created before pip fails, resulting in an orphaned temp folder within site-packages every time this command is run. This change adds a UninstallationError that is raised before the temporary folder is created, through a PermissionError exception at the utils::misc::renames level.
• Add chown warning: Installing, upgrading, and uninstalling with pip fails if users don't have access to the package folder. This often happens most often when the package is owned by root from a previous sudo pip install. This change warns the user and offers potential solutions.
• Fix pip freeze for temp folders: When orphaned temp directories are left in site-packages, pip freeze warns with a parsing error. This change makes pip ignore temp directories when running pip freeze.

Issues addressed:

• Failing fast will prevent temporary directories from being created when users run pip with the wrong user: parse error at '-irtuale', but virtualenv is correct  #7763
• The new chown warning will address this user's concerns about a permissions error: pip3 have to re-download packages after Permission denied (no sudo) #4692 Not able to install scikit-image for python #4727 Permission error with pip install -e #7940
• User concerns about ~-prefixed temp directories in pip freeze will be addressed when the warning is removed. pip3 freeze output still confusing when stray AdjacentTempDirectorys are present #7269 pip list and pip freeze treat invalid packages differently #9235

Fixes #7269
Fixes #9235

[winsonluk]

Backstory: I've made the mistake of running sudo pip install too many times, leading to a few of these issues myself. I agree with #6409 - we should probably add a warning at the command line level when users run with sudo.

[uranusjr]

I don’t think this fixes #7450, but only eliminates the subsequent failure caused by the mess it creates. To properly fix the issue, pip needs to fail on installation instead.

[winsonluk]

I don’t think this fixes #7450, but only eliminates the subsequent failure caused by the mess it creates. To properly fix the issue, pip needs to fail on installation instead.

You're right, thanks for the flag. The symptom is the same but the cause is different. #7450 relates to temp directories generated from setup.py install; this PR fixes temp directories generated from a failed pip install --upgrade after a sudo pip install. I'll unlink the issues.

[winsonluk]

FYI I've submitted a PR to CPython to fix the root cause (shutil.move() failing incorrectly): python/cpython#24001

[BrownTruck]

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will be eligible for code review and hopefully merging!

[winsonluk]

Closing since this has been merged via #9669.