Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Filename too long when invoking PEX with --pex-verbosity >=3 #11681

Closed
stuhood opened this issue Mar 12, 2021 · 2 comments · Fixed by #11711
Closed

Filename too long when invoking PEX with --pex-verbosity >=3 #11681

stuhood opened this issue Mar 12, 2021 · 2 comments · Fixed by #11711
Assignees
@jsirois
Labels
bug

Comments

@stuhood
Copy link
Member

stuhood commented Mar 12, 2021 •
edited
Loading

Any PEX-using run of Pants at 4401741 (PEX 2.1.33) with --pex-verbosity=3 (i.e. ./pants test --pex-verbosity=3 src/python/pants/util/::) fails reproducibly with a line like:

/Users/stuhood/.cache/pants/named_caches/pex_root/venvs/34bbe9579368d3ee50226ae26ab367a6fe827866/800ae8ef047bd783543e8282c22bfdbee7b7fca8/pex: File name too long

Notably: --pex-verbosity=1 does not trigger the issue.

Capturing the sandbox of the run using --no-process-execution-cleanup-local-dirs allows for reproducing the issue with the __run.sh script (have captured a repro). The complete log of ./__run.sh is attached.

ftl.log

The content of the pants_plugins.pex_pex_shim.sh file (attached as pants_plugins.pex_pex_shim.sh.txt) seems to indicate that some of the verbose output gets accidentally embedded in the generated shim script.
@stuhood
Copy link
Member Author

stuhood commented Mar 12, 2021

It looks like the computed venv_dir is capturing the verbose stdout here, and that's getting prepended onto other paths:

pants/src/python/pants/backend/python/util_rules/pex.py

Lines 792 to 799 in 7781df5

pex_request = request.pex_request
seeded_venv_request = dataclasses.replace(
pex_request, additional_args=pex_request.additional_args + ("--venv", "--seed")
)
result = await Get(BuildPexResult, PexRequest, seeded_venv_request)
# Pex --seed mode outputs the path of the PEX executable. In the --venv case this is the `pex`
# script in the venv root directory.
venv_dir = PurePath(result.result.stdout.decode().strip()).parent

@jsirois
Copy link
Contributor

jsirois commented Mar 12, 2021 •
edited by stuhood
Loading

This is a Pex bug: pex-tool/pex#1268

@jsirois jsirois self-assigned this Mar 12, 2021
jsirois added a commit to jsirois/pants that referenced this issue Mar 17, 2021
@jsirois
Upgrade to Pex 2.1.34.
3f96a0b 
This pulls in fixes for missing `pex.bin` and problems encountered when
attempting to run venv pexes with Pex debugging turned up to level
three or higher.

Release notes: https:/pantsbuild/pex/releases/edit/v2.1.34

Fixes pantsbuild#11211
Fixes pantsbuild#11681

[ci skip-rust]
[ci skip-build-wheels]
@jsirois jsirois mentioned this issue Mar 17, 2021
Merged
jsirois added a commit that referenced this issue Mar 17, 2021
@jsirois
Upgrade to Pex 2.1.34. (#11711)
8219911 
This pulls in fixes for missing `pex.bin` and problems encountered when
attempting to run venv pexes with Pex debugging turned up to level
three or higher.

Release notes: https:/pantsbuild/pex/releases/tag/v2.1.34

Fixes #11211
Fixes #11681
jsirois added a commit to jsirois/pants that referenced this issue Mar 17, 2021
@jsirois
Upgrade to Pex 2.1.34. (pantsbuild#11711)
ffe1fe2 
This pulls in fixes for missing `pex.bin` and problems encountered when
attempting to run venv pexes with Pex debugging turned up to level
three or higher.

Release notes: https:/pantsbuild/pex/releases/tag/v2.1.34

Fixes pantsbuild#11211
Fixes pantsbuild#11681
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@jsirois jsirois mentioned this issue Mar 17, 2021
Merged
jsirois added a commit to jsirois/pants that referenced this issue Mar 17, 2021
@jsirois
Upgrade to Pex 2.1.34. (pantsbuild#11711)
d9f84a2 
This pulls in fixes for missing `pex.bin` and problems encountered when
attempting to run venv pexes with Pex debugging turned up to level
three or higher.

Release notes: https:/pantsbuild/pex/releases/tag/v2.1.34

Fixes pantsbuild#11211
Fixes pantsbuild#11681

[ci skip-rust]

[ci skip-build-wheels]
@jsirois jsirois mentioned this issue Mar 17, 2021
Merged
jsirois added a commit that referenced this issue Mar 17, 2021
@jsirois
Upgrade to Pex 2.1.34. (#11711) (#11713)
59f9c30 
This pulls in fixes for missing `pex.bin` and problems encountered when
attempting to run venv pexes with Pex debugging turned up to level
three or higher.

Release notes: https:/pantsbuild/pex/releases/tag/v2.1.34

Fixes #11211
Fixes #11681
jsirois added a commit to jsirois/pants that referenced this issue Mar 17, 2021
@jsirois
Upgrade to Pex 2.1.34. (pantsbuild#11711)
d9df388 
This pulls in fixes for missing `pex.bin` and problems encountered when
attempting to run venv pexes with Pex debugging turned up to level
three or higher.

Release notes: https:/pantsbuild/pex/releases/tag/v2.1.34

Fixes pantsbuild#11211
Fixes pantsbuild#11681

(cherry picked from commit 8219911)

[ci skip-rust]

[ci skip-build-wheels]
jsirois added a commit that referenced this issue Mar 17, 2021
@jsirois
Upgrade to Pex 2.1.34. (#11711) (#11714)
10c6f45 
This pulls in fixes for missing `pex.bin` and problems encountered when
attempting to run venv pexes with Pex debugging turned up to level
three or higher.

Release notes: https:/pantsbuild/pex/releases/tag/v2.1.34

Fixes #11211
Fixes #11681

(cherry picked from commit 8219911)
Ran build-support/bin/generate_lockfile.sh
cherry-picked e3b3e53 src/python/pants/backend/python/util_rules/pex_test.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jsirois jsirois

Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Upgrade to Pex 2.1.34. jsirois/pants
2 participants
@stuhood @jsirois