[WIP] PEP 484: use package_data to distribute stubs

[tharvik]

See mypy#1895.

Change location for finding stubs from having a fixed location as shared/typehints/python* to having the stubs being directly in the module (package_data instead of data_files). Having this will tight types annotations to code rather than to repository, which IMO is good, that also remove the need to have a tree outside of /usr/lib/python*/site-packages, with version done by hand.

The type checker would still have to look in PYTHONPATH to find a suitable type definition but, as the loaded module what found in it, we don't need to check elsewhere.

[the-knights-who-say-ni]

Hello, and thanks for your contribution!

Unfortunately we couldn't find an account corresponding to your GitHub username at bugs.python.org (b.p.o). If you don't already have an account at b.p.o, please create one and make sure to add your GitHub username. If you do already have an account at b.p.o then please go there and under "Your Details" add your GitHub username.

And in case you haven't already, please make sure to sign the PSF contributor agreement (CLA); we can't legally look at your contribution until you have signed the CLA.

Once you have done everything that's needed, please reply here and someone will verify everything is in order.

[tharvik]

Unfortunately we couldn't find an account corresponding to your GitHub username at bugs.python.org (b.p.o). If you don't already have an account at b.p.o, please create one and make sure to add your GitHub username. If you do already have an account at b.p.o then please go there and under "Your Details" add your GitHub username.

Done.

And in case you haven't already, please make sure to sign the PSF contributor agreement (CLA); we can't legally look at your contribution until you have signed the CLA.

Done.

[gvanrossum]

(@berkerpeksag what happened with the "CLA not signed" label? Seems you removed it after verification but then it was added back by the bot?)

[gvanrossum]

I'm changing the subject to include "[WIP]" to indicate that this isn't ready for merging yet.

TBH I think the recommendation of searching packages using $PYTHONPATH is plain wrong and should be removed from the PEP -- $PYTHONPATH may have to be set to find the code of the type checker, which may not be using the same Python version as the code being checked. (As an extreme example, at Dropbox the target code is written in Python 2.7, while mypy itself is written in Python 3.)

I can't say anything about the recommended setup.py syntax since I have no idea what anything in a setup.py file means (and I have a suspicion that 99% of people are in the same boat, just cargo-culting setup.py magic from one project to the next without ever understanding it).

[zware]

@berkerpeksag, @gvanrossum: It looks like the CLA-signed bit hadn't been flipped on @tharvik's b.p.o profile yet when @berkerpeksag pulled the label.

[tharvik]

Then we can just remove it and leave only

Third-party stub packages can use any location for stub storage. Stub file package
authors might use the following snippet in setup.py:

I guess it was simply to find the modules and bundled stubs.

Yup, setup.py is magic (and that shouldn't be the case for good code); here goes the documentation on package_data versus data_files. IMO, stubs fit better in "data that’s closely related to the package’s implementation" than "anything which doesn’t fit in the previous categories".

[gvanrossum]

I'm sorry, I think we need to take this back to a wider audience, e.g. python-ideas. The question has two sides to it: where should a package store its stubs (if not alongside the module files), and how should mypy be able to find the stubs for a given package. There are several additional wrinkles: the Python version used to run mypy needn't be the Python version for which we want to check, and we need to avoid the problem (perhaps unique to mypy) that if you point the checker to a package in site-packages, it will look for other packages in site-packages as well.