pytest xdist should run with -n0 in debug

[elad-eyal]

when debugging a (single) test with pytest, invocation of pytest should add -n 0 . This way multiple runners will not be used and start-up time is reduced.

[eleanorjboyd]

Hi! We are looking to make changes to how we pass args so you could have multiple config types: microsoft/vscode-python#21845. Would this solve your issue? Otherwise it would need to be a separate feature request. Our general policy at this time is we are trying to edit the pytest args as little as possible since it is very hard to do right. With this in mind, I am hesistant to do a feature request that causes arg editing and this would require a larger team conversation if you do want to make this a separate feature request. Thanks

[elad-eyal]

i guess a separate python.testing.pytestArgs for debug would be a suitable workaround. i would think you'd want this behavior to be out of the box but user configuration is fine too.

cheers

[DetachHead]

ideally, it should dynamically decide how many runners to use based on how many tests are selected: pytest-dev/pytest-xdist#853

i came up with a workaround that works well with vscode's new pytest test adapter: pytest-dev/pytest-xdist#853 (comment)

// .vscode/settings.json

{
    "python.testing.pytestArgs": ["-n", "auto"]
}

# conftest.py

@hookimpl(wrapper=True)
def pytest_xdist_auto_num_workers(config: Config) -> Generator[None, int, int]:
    """determine how many workers to use based on how many tests were selected in the test explorer"""
    num_workers = yield
    if "vscode_pytest" in config.option.plugins:
        return min(num_workers, len(config.option.file_or_dir))
    return num_workers

this works because vscode passes each test to pytest as separate arguments (eg. pytest test_foo.py::test_foo test_foo.py::test_bar), so you can guess how many tests are going to run before they get collected.

edit: looks like this relies on vscode's new test adapter, which isn't yet enabled for all users. to enable it, you need to add this to your user settings.json (doesn't work in the workspace one unfortunately):

{
    "python.experiments.optInto": ["pythonTestAdapter"]
}

[paulacamargo25]

Thanks for the feature request! We are going to give the community 60 days to provide 7 👍 upvotes on the opening comment to gauge general interest in this idea. If there's enough upvotes then we will consider this feature request in our future planning. If there's unfortunately not enough upvotes then we will close this issue.

[DetachHead]

im going to try to make a PR adding this feature, because i'm pretty sure that smalll pytest_xdist_auto_num_workers hook in my workaround is all that's needed

[DetachHead]

microsoft/vscode-python#23539

[eleanorjboyd]

Thank you @DetachHead for contributing!

[shakhovski]

@DetachHead, It doesn't work with pytest 6.2.5.

2024-07-12 19:34:07.753 [error] /home/???/test_venv/lib/python3.10/site-packages/_pytest/config/__init__.py:318: PluggyTeardownRaisedWarning: A plugin raised an exception during an old-style hookwrapper teardown.
Plugin: helpconfig, Hook: pytest_cmdline_parse
AttributeError: module 'pytest' has no attribute 'Config'
...
2024-07-12 19:34:07.755 [error]     def pytest_xdist_auto_num_workers(self, config: pytest.Config) -> Generator[None, int, int]:
AttributeError: 
2024-07-12 19:34:07.755 [error] module 'pytest' has no attribute 'Config'

Commenting out your change allows Test Explorer to work.

[eleanorjboyd]

@shakhovski we are looking to no longer support any pytest version below 7, if possible it is best to update to a newer version. Thanks!