PyPI allows uploads to old versions (if you use a different format)

[hickford]

PyPI doesn't let publishers to overwrite old files: http://comments.gmane.org/gmane.comp.python.distutils.devel/22739

I've pushed changes to PyPI where it is no longer possible to reuse a filename and attempting to do it will give an 400 error that says: "This filename has previously been used, you should use a different version"

The good reasons are reliability and data integrity (see also Npm's policy http://blog.npmjs.org/post/77758351673/no-more-npm-publish-f)

PyPI is more complex than Npm because you can upload multiple formats under one version. Right now you can delete a tar ball package-1.2.1.tar.gz and upload a zip package-1.2.1.zip to the same version. This has the same potential to shaft users (accidentally or maliciously) who rely on a specific version.

Perhaps PyPI's policy should be strengthened to 'no uploads to old versions' (say, older than 1 day)

What do you think?

[qwcode]

+1

[domibarton]

I think that behaviour is quite OK for the live repo.

Though, to be honest, it's a huge PITA for the test repo. I support integrity and all that stuff on "production" systems. However, developers need to have their code / packages checked somewhere and it's a PITA if you can't upload them same version twice while testing a new release.

There's no other way than the test repo to test your package. With git (or any other SCM) you can easily create a new branch and test it until you're sure everything works. Or if you've a look at PHP Packagist (compose) there's a -dev version for each development branch. On Docker the same, you can test your feature/release branches before tagging and going "live".

With the new policy you basically say: You've ONE SINGLE TRY and that one SHOULD WORK. No chance for a 2nd try. IMHO this isn't the purpose of a testing system and breaks the whole "we've a testing repo" idea. To be honest, I think this only leads to annoyed developers and a lot of "crippled versions" because developers couldn't properly test their versions before going live.

tl;dr:
I suppose you do that on the live system but not on the test system.

[ncoghlan]

I turned @domibarton's comment into a Warehouse RFE: pypi/warehouse#868

[ncoghlan]

@dstufft pointed out there was already an issue tracking that proposal: pypi/warehouse#720

[domibarton]

@ncoghlan Thanks! I'll have a look at it and maybe I can help & support you guys somewhere.

[ncoghlan]

In August 2016, PyPI's release policy was modified to only allow exactly one sdist (regardless of format) per release: https://www.python.org/dev/peps/pep-0527/#limiting-number-of-sdists-per-release

We've made the deliberate choice to allow wheel files to be added to old releases, though, and advise folks to use --no-binary and build their own wheel files from source if that is a concern.

While I believe we do guard against this, if there's a loophole in the artifact hash checking where deleting the old sdist allows you to upload a new one with a different file extension, that should be filed as a security bug directly against https:/pypa/warehouse

[thibaudcolas]

Edit: I see there’s lots more discussion of this in #112, and potentially more on the mailing list, so I’ll follow that thread, and edit my comment when I found more info. Sorry for the distraction!

Edit edit – after reading #112, pypi/warehouse#720 (comment), and also pypa/pip#5874 (comment), I think the takeaway is that there are ways for users concerned by this to opt into stricter installs (with the appropriate pip flags, or a more opinionated installer that uses hashes by default like Pipenv/Poetry).

---

I know it’s been a while since this has been discussed but can anyone expand on why that behaviour is considered to be ok for the live PyPI, and why you’ve made the choice to allow wheels for old releases?

I can see the potential for abuse, but also the convenience of publishing wheels for old releases (see also meshy/pythonwheels#19). I know PyPI doesn’t offer much guarantees when it comes to the security of the packages published to it, but it does prevent reuploading the same artifact multiple times, and to me uploading a new wheel feels equally as dangerous (maliciously or not) since I presume a lot of people use installers that would default to wheels if available, without any lockfile / checksum.

[ncoghlan]

@thibaudcolas That's right - adding a wheel will still trigger a hash check failure, and it isn't invisible the way artifact replacement is (both the sdist and the new wheel are shown in the UI and API responses)