Dependencies getting reinstalled despite no changes and recreate=False

[bittner]

I have a relatively simple Django-based CLI application with Polars as a second main dependency. It's packaged with setuptools and automatic versioning using setuptools-scm. The project uses Git LFS for some files (test fixtures), if that makes a difference.

When I run my unit tests using Tox the package dependencies always get reinstalled, even if nothing changed, and I wonder why.

$ tox
.pkg: _optional_hooks> python D:\dev\foobar\venv\lib\site-packages\pyproject_api\_backend.py True setuptools.build_meta 
.pkg: get_requires_for_build_sdist> python D:\dev\foobar\venv\lib\site-packages\pyproject_api\_backend.py True setuptools.build_meta 
.pkg: build_sdist> python D:\dev\foobar\venv\lib\site-packages\pyproject_api\_backend.py True setuptools.build_meta 
py: install_package> python -I -m pip install --force-reinstall --no-deps D:\dev\foobar\.tox\.tmp\package\16\foobar-2.4.1.dev13.tar.gz 
py: commands[0]> coverage run -m pytest . 
=================================================== test session starts =================================================== 
platform win32 -- Python 3.9.13, pytest-8.2.2, pluggy-1.5.0 -- D:\dev\foobar\.tox\py\Scripts\python.EXE
cachedir: .tox\py\.pytest_cache
django: version: 4.2.13, settings: application.settings (from ini)
rootdir: D:\dev\foobar
configfile: pyproject.toml
plugins: django-4.8.0
collecting ... collected 42 items 

...  (test output omitted)

------------------------------- generated xml file: D:\dev\foobar\tests\junit-report.xml ---------------------------------- 
=================================================== 42 passed in 4.30s ==================================================== 
py: commands[1]> coverage report
Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
foobar\tests\conftest.py      48      1    98%   71
foobar\views.py               51      6    88%   32-51
-------------------------------------------------------
TOTAL                      1216      7    99%

25 files skipped due to complete coverage.
py: commands[2]> coverage xml
Wrote XML report to coverage.xml
  py: OK (252.58=setup[245.67]+cmd[5.84,0.52,0.55] seconds)
  congratulations :) (252.80 seconds)

The build takes more than 4 minutes, mostly because it runs on a corporate Windows machine with an "Antimalware Service Executable" severely interfering (~ 25% CPU load alone). Most of the time is spent in the py: install_package> step.

Note the python -I -m pip install --force-reinstall. This is done always, no matter if I make changes to the source code or not. Even if I set recreate=False explicitly in the Tox configuration. Shouldn't the virtual environment not be repopulated in this case, to save time?

What can I do to make Tox/Pip stop reinstalling dependencies when this is apparently not needed? Is there something wrong with my setup, maybe?

Project Setup

tox.ini (click to expand)

[tox]
envlist = py

[testenv]
description = Unit tests
deps =
    cli-test-helpers
    coverage[toml]
    pytest-django
commands =
    coverage run -m pytest {posargs:.}
    coverage report
    coverage xml
setenv =
    DJANGO_DEBUG=false
    DJANGO_TESTING=pytest

pyproject.toml (click to expand)

[build-system]
build-backend = "setuptools.build_meta"
requires = ["setuptools>=64", "setuptools_scm>=8"]

[project]
name = "foobar"
dynamic = ["version"]
description = "An application developed in tedious infrastructure"
readme = "README.md"
license = {file = "LICENSE"}
authors = [
  {name = "John Doe", email = "[email protected]"},
]
classifiers = [
  "Development Status :: 5 - Production/Stable",
  "Framework :: Django",
  "Intended Audience :: Information Technology",
  "License :: Other/Proprietary License",
  "Programming Language :: Python :: 3",
  "Programming Language :: Python :: 3.9",
]
keywords = [
  "django",
]
requires-python = ">=3.9"
dependencies = [
  "cx-oracle",
  "dj-database-url",
  "django==4.2.*",
  "gunicorn[gthread]",
  "polars",
  "python-decouple",
]

[project.optional-dependencies]
dev = [
  "django-extensions",
  "ipython",
]

[project.scripts]
"foobar" = "application.manage:main"

[project.urls]
Homepage = "https://wiki.intranet.corp/display/foobar/"

[tool.coverage.report]
exclude_also = [
  "if __name__ == .__main__.:",
  "    INSTALLED_APPS.append..behave_django..",
  "    INSTALLED_APPS.append..django_extensions..",
]
show_missing = true
skip_covered = true

[tool.coverage.run]
source = ["."]
omit = [
  "*/migrations/*",
  "tests/*",
  "tests.py",
  "test_*.py",
  "*_tests.py",
]

[tool.pytest.ini_options]
addopts = "--color=yes --doctest-modules --ignore=tests --junitxml=tests/junit-report.xml --verbose"
python_files = ["tests.py", "test_*.py", "*_tests.py"]
DJANGO_SETTINGS_MODULE = "application.settings"
FAIL_INVALID_TEMPLATE_VARS = true

[tool.setuptools.packages.find]
namespaces = true

[tool.setuptools_scm]
local_scheme = "no-local-version"

MANIFEST.in (click to expand)

graft */static
graft */templates
global-exclude *.py[cod] *_tests.py test_*.py tests.py
exclude [Dd]ocker* Jenkinsfile* MANIFEST.in tox.ini .env *.env *.md *.yml .*config .*ignore .*.yml
prune .git*
prune **/tests

[gaborbernat]

The package needs to be rebuilt every time, so you're actually testing the library as is. The library dependent list don't get reinstalled, however the library itself doesn't need to be rebuilt every time and reinstalled.