pip 20.2 - ValueError exception on improperly formatted package

[cjp256]

Environment

• pip version: 20.2
• Python version: 3.6
• OS: Ubuntu 18.04

Description

pip install now fails to install improperly packaged python project. The project has installed a file to 'purelib', rather than under the 'purelib' directory. While the broken project needs to be fixed, pip is now throwing an exception without a hint.

This change in behavior is from #8562 (#8562 (comment)).

Expected behavior

Prior to 20.2, pip install worked fine. The purelib file was not required and was seemingly ignored.

How to Reproduce

Using Ubuntu 18.04:
pip install -v http://archive.ubuntu.com/ubuntu/pool/main/p/python-apt/python-apt_1.6.5ubuntu0.2.tar.xz

Output

ERROR: Exception:
Traceback (most recent call last):
  File "/tmp/venv/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
    status = self.run(options, args)
  File "/tmp/venv/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
    return func(self, options, args)
  File "/tmp/venv/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 421, in run
    pycompile=options.compile,
  File "/tmp/venv/lib/python3.6/site-packages/pip/_internal/req/__init__.py", line 90, in install_given_reqs
    pycompile=pycompile,
  File "/tmp/venv/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 831, in install
    requested=self.user_supplied,
  File "/tmp/venv/lib/python3.6/site-packages/pip/_internal/operations/install/wheel.py", line 830, in install_wheel
    requested=requested,
  File "/tmp/venv/lib/python3.6/site-packages/pip/_internal/operations/install/wheel.py", line 658, in _install_wheel
    for file in files:
  File "/tmp/venv/lib/python3.6/site-packages/pip/_internal/operations/install/wheel.py", line 587, in make_data_scheme_file
    _, scheme_key, dest_subpath = normed_path.split(os.path.sep, 2)
ValueError: not enough values to unpack (expected 3, got 2)

In this case, normed_path is python_apt-0.0.0.data/purelib, and points to a file (not a directory).

[chrahunt]

To add to the reproduction instructions:

cd "$(mktemp -d)"
python -m venv .env
./.env/bin/python -m pip install --upgrade pip
./.env/bin/python -m pip install wheel
sudo apt install libapt-pkg-dev
./.env/bin/python -m pip install -v http://archive.ubuntu.com/ubuntu/pool/main/p/python-apt/python-apt_1.6.5ubuntu0.2.tar.xz

which results in

ERROR: Exception:
Traceback (most recent call last):
  File "/tmp/user/1000/tmp.wTL19RjBCa/.env/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
    status = self.run(options, args)
  File "/tmp/user/1000/tmp.wTL19RjBCa/.env/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
    return func(self, options, args)
  File "/tmp/user/1000/tmp.wTL19RjBCa/.env/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 412, in run
    installed = install_given_reqs(
  File "/tmp/user/1000/tmp.wTL19RjBCa/.env/lib/python3.8/site-packages/pip/_internal/req/__init__.py", line 82, in install_given_reqs
    requirement.install(
  File "/tmp/user/1000/tmp.wTL19RjBCa/.env/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 823, in install
    install_wheel(
  File "/tmp/user/1000/tmp.wTL19RjBCa/.env/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py", line 821, in install_wheel
    _install_wheel(
  File "/tmp/user/1000/tmp.wTL19RjBCa/.env/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py", line 657, in _install_wheel
    for file in files:
  File "/tmp/user/1000/tmp.wTL19RjBCa/.env/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py", line 586, in make_data_scheme_file
    _, scheme_key, dest_subpath = normed_path.split(os.path.sep, 2)
ValueError: not enough values to unpack (expected 3, got 2)