Keep/drop/update PYTHONPATH page?

[effigies]

In my experience, modifying the PYTHONPATH only leads to headaches, so I would be disinclined to encourage people to use it. On the other hand, it could be useful to know about for debugging messy environments.

[oesteban]

+1 to drop it

[matthew-brett]

I seem to remember having the same reluctance when I wrote that section originally - but - have you got a better suggestion for an easy-to-explain way of putting some Python code on the Python path? I wonder whether we could start with PYTHONPATH, as here, because it is very easy to explain, then later move to say something like a tiny Python package with pyproject.toml, and explain they should always do that, and not use PYTHONPATH.

[oesteban]

have you got a better suggestion for an easy-to-explain way of putting some Python code on the Python path?

Yes, creating the files in the current working directory.

I see PYTHONPATH as part of the packaging. Although it has nothing to do with packaging, you really need to understand python packaging correctly (and environments, venvs, etc.) to make safe use of PYTHONPATH.

[matthew-brett]

Well - the problem there is that they always have to work in the same working directory to find their code.

The problem gets obscured often if you use notebooks all the time, because you get used to not refactoring your code out into functions, but that's partly what I was trying to avoid here.

I see PYTHONPATH as a primitive way of doing the job - but it's quite a good way of helping people to think about what's going on - that the package directories need to by on the Python sys.path. And that in turn helps them think a bit less magically about packaging - as in "pip / conda will do all this for me, no need to worry". Because of course that makes it a much bigger leap to refactoring into your own packages.

Of PYTHONPATH, being primitive, is not the right way to do this when you've got started, but I think we can cover that.

[oesteban]

Honestly, IMHO it is a deviation from the course's focus that is definitely not worth the pain:

• Python environments are complicated, Python packaging is complicated, and PYTHONPATH is just another layer of that. We ask them to be able to write a for loop (mostly of them coming from Matlab), why bother with those complications?
• Importing Python code that is located in the notebook's directory is very obscure (e.g., that rotations.py file) - but it is also unnecessary because that code is something you want the students to actually read, so embed it in the notebook.

Talking about this without a single example where we have to run three mortal twists in the background to avoid having students playing with PYTHONPATH seems unnecessary to me.

[matthew-brett]

Absolutely - if they were always going to be working in the notebook, and we didn't worry to much about helping them not work in the notebook, then there would be little point in discussing stuff like packages and paths ...

Of course - that is the stuff you see, mainly, in the current book, but even early on I focus on getting them to write tests and put functions into modules. Then their projects are to write code in modules and scripts to do outlier detection, and then, analyze an OpenFMRI study ...

[oesteban]

In that case, I'd give them some stub structure of Python files and tell them where to write their stuff.

To me, it is more important to teach the logic of how you better structure code (in general). Then, making it digestible by the Python interpreter (in particular) comes naturally (and yes, then you may indicate someone at a given instance that modifying the PYTHONPATH is the fastest way - but I would avoid making it part of the general discourse of your teaching materials).

[effigies]

Would the following do most of what people need to just get boostrapped? It would limit you to importing local (presumably related) code and not infect environments.

import sys
import os

# Enable importing from the local directory until I turn this into a proper module
sys.path.insert(0, os.path.dirname(__file__))

The worry I have is that I write something like:

code/
  mylib.py
  myscript.py

And then later I make a package mylib and don't notice that myscript (or some copy of it) is still importing from code/mylib instead of $SITEPACKAGES/mylib.

[matthew-brett]

Yes, that's not a bad idea (the sys.path idea) - in that it is easy to explain again, and sufficiently ugly that any reasonable person will want to get rid of it as soon as possible!