poetryup fails to execute on Windows within a Poetry environment

[johnthagen]

> poetry shell
> poetryup --latest
Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\User\AppData\Local\pypoetry\Cache\virtualenvs\tmp-N1D6edvU-py3.10\Scripts\poetryup.exe\__main__.py", line 7, in <module>
  File "C:\Users\User\AppData\Local\pypoetry\Cache\virtualenvs\tmp-N1D6edvU-py3.10\lib\site-packages\typer\main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "C:\Users\User\AppData\Local\pypoetry\Cache\virtualenvs\tmp-N1D6edvU-py3.10\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\User\AppData\Local\pypoetry\Cache\virtualenvs\tmp-N1D6edvU-py3.10\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "C:\Users\User\AppData\Local\pypoetry\Cache\virtualenvs\tmp-N1D6edvU-py3.10\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\User\AppData\Local\pypoetry\Cache\virtualenvs\tmp-N1D6edvU-py3.10\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\User\AppData\Local\pypoetry\Cache\virtualenvs\tmp-N1D6edvU-py3.10\lib\site-packages\typer\main.py", line 500, in wrapper
    return callback(**use_params)  # type: ignore
  File "C:\Users\User\AppData\Local\pypoetry\Cache\virtualenvs\tmp-N1D6edvU-py3.10\lib\site-packages\poetryup\main.py", line 47, in poetryup
    pyproject = Pyproject(pyproject_str)
  File "C:\Users\User\AppData\Local\pypoetry\Cache\virtualenvs\tmp-N1D6edvU-py3.10\lib\site-packages\poetryup\core\pyproject.py", line 26, in __init__
    self.poetry_version = version_.parse(self.__get_poetry_version())
  File "C:\Users\User\AppData\Local\pypoetry\Cache\virtualenvs\tmp-N1D6edvU-py3.10\lib\site-packages\poetryup\core\pyproject.py", line 327, in __get_poetry_version
    subprocess.run(
  File "C:\Program Files\Python310\lib\subprocess.py", line 501, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Program Files\Python310\lib\subprocess.py", line 966, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Program Files\Python310\lib\subprocess.py", line 1435, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified

It seems that on Windows, poetryup cannot find the path to poetry when installed in a Poetry session. This works fine when I tested it on macOS. Perhaps there is some kind of environment variable/subprocess issue when poetryup is running on Windows?

Environment

> python --version
Python 3.10.3

> poetry --version
Poetry version 1.1.13

> pip list
Package    Version
---------- -------
click      8.1.3
colorama   0.4.5
packaging  21.3
pip        22.1
poetryup   0.8.1
pyparsing  3.0.9
setuptools 62.2.0
tomlkit    0.11.0
typer      0.4.1
wheel      0.37.1

[tool.poetry]
name = "tmp"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.10"

[tool.poetry.dev-dependencies]
poetryup = "^0.8.1"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[MousaZeidBaker]

I think it's related to subprocess.

Can you spawn a python shell and try to run the following (remember import subprocess)

• subprocess.run(["poetry", "--version"])
• subprocess.run("poetry --version", shell=True)

[johnthagen]

@MousaZeidBaker Yep, seems like that exactly.

This was performed within the poetry shell session with the same information as the initial comment in this post:

> python
Python 3.10.3 (tags/v3.10.3:a342a49, Mar 16 2022, 13:07:40) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.run(["poetry", "--version"])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\Python310\lib\subprocess.py", line 501, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Program Files\Python310\lib\subprocess.py", line 966, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Program Files\Python310\lib\subprocess.py", line 1435, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
>>> subprocess.run("poetry --version", shell=True)
Poetry version 1.1.13
CompletedProcess(args='poetry --version', returncode=0)

[MousaZeidBaker]

Okay then shell=True solves the issue however it's discouraged to use it, see security considerations. Also Python docs states

The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell

So how are you installing poetry? Is it available in all your shells?

[johnthagen]

I install Poetry using the official installer. I agree it's odd to need shell=True. I presume the issue has to do with poetry shell creating a new shell instance. That must somehow effect subprocesses ability to see the path to poetry?

[johnthagen]

@MousaZeidBaker Would you consider conditionally setting shell=True only when run on Windows?

Something like:

shell = platform.system() == "Windows"

Added to:

poetryup/src/poetryup/core/cmd.py

Lines 27 to 31 in b7591d9

	process = subprocess.run(
	cmd,
	stdout=subprocess.PIPE if capture_output else None,
	stderr=subprocess.STDOUT if capture_output else None,
	)

This minimizes the usage of shell=True while still supporting that common workflow of declaring, locking, and managing all dev dependencies within Poetry's environment in a cross-platform way.

I'd really like to use poetryup, but need to manage it within a Poetry environment to ensure locking and a consistent experience for developers across machines & operating systems.

[johnthagen]

Brainstorming, another possible idea would be to make poetryup a Poetry plugin:

• https://python-poetry.org/docs/plugins#plugins

This would avoid shelling out to poetry and would probably provide a more native experience for all users?

[johnthagen]

I tried this on Poetry 1.2.x on Windows and poetryup was able to call poetry without shell=True. I suspect this is due to the new installer for Poetry 1.2.x installing differently than the old get-poetry method.

In any case, this doesn't seem to be an issue with poetryup.

[MousaZeidBaker]

Glad to hear this issue is finally resolved without shell=True. Great detective work, yes probably due to the new installer.

Whenever I get some time over I'll have a look into poetry plugin as it sounds as a good option. Thanks for your contribution.

[MousaZeidBaker]

Just wanted to let you know poetryup is now a plugin with additional features, see poetry-plugin-up