Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VS Code executable not found after downloading it #246

Closed
DonJayamanne opened this issue Nov 21, 2023 · 25 comments · Fixed by #247 or #248
Closed

VS Code executable not found after downloading it #246

DonJayamanne opened this issue Nov 21, 2023 · 25 comments · Fixed by #247 or #248
Assignees
Labels
bug Issue identified by VS Code Team member as probable bug
Milestone

Comments

@DonJayamanne
Copy link

DonJayamanne commented Nov 21, 2023

Does this issue occur when all extensions are disabled?: Yes/No

  • VS Code Version: Insiders
  • OS Version: Windows

CI logs https:/microsoft/vscode-jupyter/actions/runs/6940223424/job/18878834977
I am downloading VS Code using the package @vscode/test-electron

I have noticed that the windows tests can fail sometimes due to some file not found error (see below).
Re-running the CI seems to fix it (hence the reason I think its flaky)

Please let me know if additional information/logs are required.

Here are the logs from the Jupyter CI

Run GabrielBB/[email protected]
C:\Windows\system32\cmd.exe /D /S /C "C:\hostedtoolcache\windows\node\18.15.0\x64\npm.cmd run testSmokeLogged"

> [email protected] testSmokeLogged
> cross-env VSC_JUPYTER_FORCE_LOGGING=true node --no-force-async-hooks-checks ./out/test/testBootstrap.node.js ./out/test/smokeTest.node.js

Starting socket server for tests.
Creating test server
Listening to test server
Test server running
Test server listening on port 53222
Spawning C:\hostedtoolcache\windows\node\18.15.0\x64\node.exe : ./out/test/smokeTest.node.js
Python used in tests is c:\hostedtoolcache\windows\python\3.10.11\x64\python.exe
Start Test Runner
Launch tests in test runner
****************************************************************************************************
Start Standard tests
Downloading VS Code insiders from https://update.code.visualstudio.com/latest/win32-x64-archive/insider
Downloading VS Code (131091780B)
Downloading VS Code [=-----------------------------] 6%Downloading VS Code [=====-------------------------] 17%Downloading VS Code [=========---------------------] 33%Downloading VS Code [==============----------------] 48%Downloading VS Code [===================-----------] 64%Downloading VS Code [====================----------] 69%Downloading VS Code [=====================---------] 71%Downloading VS Code [======================--------] 74%Downloading VS Code [=======================-------] 77%Downloading VS Code [=======================-------] 80%Downloading VS Code [=========================-----] 85%Downloading VS Code [==========================----] 90%Downloading VS Code [============================--] 94%Downloaded VS Code into D:\a\vscode-jupyter\vscode-jupyter\.vscode-test\vscode-win32-x64-archive-insiders
Downloaded VS Code into D:\a\vscode-jupyter\vscode-jupyter\.vscode-test\vscode-win32-x64-archive-insiders
Installing Python Extension ms-python.python to C:\Users\RUNNER~1\AppData\Local\Temp\vscode_jupyter_exts
Installing Pylance Extension to C:\Users\RUNNER~1\AppData\Local\Temp\vscode_jupyter_exts
Installing Renderer Extension to C:\Users\RUNNER~1\AppData\Local\Temp\vscode_jupyter_exts
Test error: Error: spawn D:\a\vscode-jupyter\vscode-jupyter\.vscode-test\vscode-win32-x64-archive-insiders\Code - Insiders.exe ENOENT
Exit code:   -4058
End Standard tests (with errors) Failed
****************************************************************************************************
Start Standard tests
End Standard tests (with errors) Error: ENOENT: no such file or directory, open 'D:\a\vscode-jupyter\vscode-jupyter\.vscode-test\vscode-win32-x64-archive-insiders\resources\app\product.json'
    at Object.openSync (node:fs:601:3)
    at Object.readFileSync (node:fs:469:[35](https:/microsoft/vscode-jupyter/actions/runs/6940223424/job/18878834977#step:13:36))
    at Object.insidersDownloadDirMetadata (D:\a\vscode-jupyter\vscode-jupyter\node_modules\@vscode\test-electron\out\util.js:114:41)
    at download (D:\a\vscode-jupyter\vscode-jupyter\node_modules\@vscode\test-electron\out\download.js:281:72)
    at downloadAndUnzipVSCode (D:\a\vscode-jupyter\vscode-jupyter\node_modules\@vscode\test-electron\out\download.js:[36](https:/microsoft/vscode-jupyter/actions/runs/6940223424/job/18878834977#step:13:37)1:18)
    at start (D:\a\vscode-jupyter\vscode-jupyter\out\test\standardTest.node.js:144:83)
    at D:\a\vscode-jupyter\vscode-jupyter\out\test\standardTest.node.js:177:16
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  errno: -[40](https:/microsoft/vscode-jupyter/actions/runs/6940223424/job/18878834977#step:13:41)[58](https:/microsoft/vscode-jupyter/actions/runs/6940223424/job/18878834977#step:13:59),
  syscall: 'open',
  code: 'ENOENT',
  path: 'D:\\a\\vscode-jupyter\\vscode-jupyter\\.vscode-test\\vscode-win32-x[64](https:/microsoft/vscode-jupyter/actions/runs/6940223424/job/18878834977#step:13:65)-archive-insiders\\resources\\app\\product.json'
}
Tests Exited with code 1
Error in running Smoke Tests Failed with code 1.
Exiting with test failures
Killing VSC
Error: The process 'C:\hostedtoolcache\windows\node\18.15.0\x64\npm.cmd' failed with exit code 1
@DonJayamanne
Copy link
Author

The second run with unrelated changes resulted in a clean run without any issues
https:/microsoft/vscode-jupyter/actions/runs/6940290925/job/18879022414?pr=14747

Here are two runs on on the same commit in CI:

@DonJayamanne
Copy link
Author

Hmm, ran into this exact same issue on MacOS on the CI server
This time with the latest version of @vscode/test-electron

https:/microsoft/vscode-jupyter/actions/runs/6951074099/job/18912450458

Still trying to figure out whats going on here.

@DonJayamanne DonJayamanne transferred this issue from microsoft/vscode Nov 22, 2023
@DonJayamanne DonJayamanne changed the title Flaky vs code tests on Windows VS Code executable not found after downloading it Nov 22, 2023
@DonJayamanne
Copy link
Author

DonJayamanne commented Nov 22, 2023

Looks like solmething is very wrong with the way files get extracted
https:/microsoft/vscode-jupyter/actions/runs/6951254070/job/18913278166?pr=14758

Downloaded VS Code into D:\a\vscode-jupyter\vscode-jupyter\.vscode-test\vscode-win32-x64-archive-insiders
VS Code executable D:\a\vscode-jupyter\vscode-jupyter\.vscode-test\vscode-win32-x64-archive-insiders\Code - Insiders.exe exists = false
Files in folder D:\a\vscode-jupyter\vscode-jupyter\.vscode-test\vscode-win32-x64-archive-insiders => af.pak
am.pak
app
ar.pak
bg.pak
bn.pak
ca.pak
cs-cz
cs.pak
da.pak
de-de
de.pak
el.pak
en-GB.pak
en-us
en-US.pak

The files are in the wrong place.
The files *.pak should be in D:\a\vscode-jupyter\vscode-jupyter\.vscode-test\vscode-win32-x64-archive-insiders\locales and not Files in folder D:\a\vscode-jupyter\vscode-jupyter\.vscode-test\vscode-win32-x64-archive-insiders

@DonJayamanne
Copy link
Author

Found that in all of the cases downloading never goes to 100%.
I.e. download stops before getting to 100%, thus the stream is no complete, hence extraction of the file fails.

@connor4312
Copy link
Member

connor4312 commented Nov 22, 2023

Because this is happening across different environments in your CI, and seemingly(?) only in your CI as we've not had other reports of the issue, I suspect something might be up with the archives that get downloaded. This is supported by

  • seeing the issue on multiple platforms, which use different unzip implementations (on macOS we use the command-line unzip utility, on windows we use the jszip module.)
  • I'm assuming seeing this issue come up all of a sudden without package changes, whereas we have changed CDNs

Currently we don't do any validation on downloaded archives, so I'll add some.

@DonJayamanne
Copy link
Author

Because this is happening across different environments in your CI, and seemingly(?) only in your CI as we've not had other

Logan and Steven came across this in their pipeline too.
@lramos15 /cc

@connor4312
Copy link
Member

connor4312 commented Nov 22, 2023

I believe still it's a good theory with the two bullet points I mentioned.

I made a fix to try to solve that, please validate it once published. Note that hash validation is only available when you ask for a specific commit in the version you pass (unless Joao tweaks the update server before I publish in the morning) so if you do still see the issue, please try passing a specific commit and seeing if that avoids the issue.

@DonJayamanne
Copy link
Author

Still running into this on CI, (with the latest npm package)
This time on Mac CI
https:/microsoft/vscode-jupyter/actions/runs/6962842187/job/18947373636

Start Standard tests
Downloading VS Code insiders from https://update.code.visualstudio.com/latest/darwin/insider
Downloading VS Code (130406165B)
Downloading VS Code [------------------------------] 3%Downloading VS Code [=-----------------------------] 3%Downloading VS Code [=-----------------------------] 4%Downloading VS Code [=-----------------------------] 6%Downloading VS Code [==----------------------------] 9%Downloading VS Code [===---------------------------] 13%Downloading VS Code [====--------------------------] 15%Downloading VS Code [=====-------------------------] 17%Downloading VS Code [=====-------------------------] 17%Downloading VS Code [=====-------------------------] 18%Downloading VS Code [======------------------------] 20%Downloading VS Code [======------------------------] 21%Downloading VS Code [======------------------------] 22%Downloading VS Code [======------------------------] 23%Downloading VS Code [=======-----------------------] 25%Downloading VS Code [=======-----------------------] 27%Downloading VS Code [========----------------------] 28%Downloading VS Code [========----------------------] 29%Downloading VS Code [========----------------------] 30%Downloading VS Code [=========---------------------] 31%Downloading VS Code [=========---------------------] 33%Downloading VS Code [==========--------------------] 35%Downloading VS Code [==========--------------------] 35%Downloading VS Code [==========--------------------] 36%Downloading VS Code [===========-------------------] 39%Downloading VS Code [===========-------------------] 39%Downloading VS Code [============------------------] 41%Downloading VS Code [=============-----------------] 44%Downloading VS Code [=============-----------------] 44%Downloading VS Code [==============----------------] 48%Downloading VS Code [===============---------------] [51](https:/microsoft/vscode-jupyter/actions/runs/6962842187/job/18947373636#step:13:52)%Downloading VS Code [===============---------------] [52](https:/microsoft/vscode-jupyter/actions/runs/6962842187/job/18947373636#step:13:53)%Downloading VS Code [================--------------] 54%Downloading VS Code [=================-------------] 59%Downloading VS Code [===================-----------] 64%Downloading VS Code [====================----------] 68%Downloading VS Code [====================----------] 68%Downloading VS Code [=====================---------] 71%Downloading VS Code [=====================---------] 73%Downloading VS Code [======================--------] 74%Downloading VS Code [=======================-------] 78%Downloading VS Code [========================------] 82%Downloading VS Code [========================------] 82%Downloading VS Code [=========================-----] 84%Downloading VS Code [=========================-----] 87%Downloading VS Code [==========================----] 87%Downloading VS Code [==========================----] 90%Downloading VS Code [===========================---] 93%Downloading VS Code [============================--] 96%Downloading VS Code [=============================-] 100%Downloaded VS Code into /Users/runner/work/vscode-jupyter/vscode-jupyter/.vscode-test/vscode-darwin-insiders
Downloaded VS Code into /Users/runner/work/vscode-jupyter/vscode-jupyter/.vscode-test/vscode-darwin-insiders
Installing Python Extension ms-python.python to /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/vscode_jupyter_exts
Installing Pylance Extension to /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/vscode_jupyter_exts
Installing Renderer Extension to /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/vscode_jupyter_exts
Test error: Error: spawn /Users/runner/work/vscode-jupyter/vscode-jupyter/.vscode-test/vscode-darwin-insiders/Visual Studio Code - Insiders.app/Contents/MacOS/Electron ENOENT
Exit code:   -2
End Standard tests (with errors) Failed
****************************************************************************************************
Start Standard tests
End Standard tests (with errors) Error: ENOENT: no such file or directory, open '/Users/runner/work/vscode-jupyter/vscode-jupyter/.vscode-test/vscode-darwin-insiders/Visual Studio Code - Insiders.app/Contents/Resources/app/product.json'
    at Object.openSync (node:fs:[60](https:/microsoft/vscode-jupyter/actions/runs/6962842187/job/18947373636#step:13:61)1:3)
    at Object.readFileSync (node:fs:469:35)

@DonJayamanne
Copy link
Author

Will try a specific commit today

@alexr00
Copy link
Member

alexr00 commented Nov 23, 2023

I'm also hitting this. Trying with the latest npm package now...

@alexr00 alexr00 added this to the November 2023 milestone Nov 23, 2023
@alexr00
Copy link
Member

alexr00 commented Nov 23, 2023

No luck. I'm able repro locally too. I can see that D:\repos\Microsoft\vscode-pull-request-github\.vscode-test\vscode-win32-x64-archive-insiders\Code - Insiders.exe simply doesn't exist.

@alexr00 alexr00 reopened this Nov 23, 2023
@alexr00
Copy link
Member

alexr00 commented Nov 23, 2023

Rerunning CI has not fixed it for me. Maybe interesting: so far, this has only occurred in my public PR CI and locally, not in the internal CI that I have for releasing builds.

Edit: actually after 5 reruns it's succeeded.

@connor4312
Copy link
Member

connor4312 commented Nov 23, 2023

@alexr00 can you confirm it happens when you request the version via commit hash so that it did the SHA verification after download?

@alexr00
Copy link
Member

alexr00 commented Nov 23, 2023

Yes, I set the version here to the latest insiders commit. I only tried this locally though, I didn't try this in the pipeline.

@connor4312
Copy link
Member

Thanks, will try to repro and investigate more after the holiday.

@joaomoreno
Copy link
Member

joaomoreno commented Nov 24, 2023

This is too coincidental with happening at the same time we're rolling out the new CDN to insider builds. Investigating...

BTW, update service now has the X-SHA256 changes: https://dev.azure.com/monacotools/Monaco/_build/results?buildId=246022&view=results

@joaomoreno joaomoreno self-assigned this Nov 24, 2023
@joaomoreno joaomoreno added the bug Issue identified by VS Code Team member as probable bug label Nov 24, 2023
@joaomoreno
Copy link
Member

Tests were only running against stable. I've added tests for insiders, which are now failing:

Image

@joaomoreno
Copy link
Member

Found the issue. It's indeed related to the new CDN.

We're relying on the Content-Type header to detect whether the download is a ZIP or a TGZ:

const isZip = contentType ? contentType === 'application/zip' : url.endsWith('.zip');

The new CDN unfortunately always sends application/octet-stream:

➜  ~ http -F -h https://update.code.visualstudio.com/latest/darwin-arm64/insider
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 12449
ApiVersion: Distribute 1.1
Cache-Control: public, max-age=86400
Content-Disposition: attachment; filename=VSCode-darwin-arm64.zip; filename*=UTF-8''VSCode-darwin-arm64.zip
Content-Length: 123910672
Content-Type: application/octet-stream
Date: Fri, 24 Nov 2023 10:43:49 GMT
Etag: "0xA44299607ACF10F270F36D79ADF383369949B4C0BE16DEAFEC16D8071E72B32A"
Last-Modified: Fri, 24 Nov 2023 06:29:55 GMT
Server: ECAcc (mil/6BBD)
X-Cache: HIT

➜  ~ http -F -h https://update.code.visualstudio.com/latest/darwin-arm64/stable
HTTP/1.1 200 OK
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: x-ms-request-id,x-ms-version,x-ms-lease-status,x-ms-lease-state,x-ms-blob-type
Age: 1268675
Cache-Control: max-age=31536000, public
Content-Disposition: attachment; filename="VSCode-darwin-arm64.zip"
Content-Length: 125047073
Content-MD5: g75j7rfjdHvKVTNfWLbMQg==
Content-Type: application/zip
Date: Fri, 24 Nov 2023 10:43:54 GMT
Etag: "0x8DBE11687A6A0BA"
Last-Modified: Thu, 09 Nov 2023 11:25:08 GMT
Server: ECAcc (mil/6C9A)
X-Cache: HIT
x-ms-blob-type: BlockBlob
x-ms-lease-state: available
x-ms-lease-status: unlocked
x-ms-request-id: e891b733-401e-0035-0d39-13490d000000
x-ms-version: 2013-08-15

This explains why Windows and macOS are broken, while Linux isn't: the ZIP archives are incorrectly "detected" as TGZ.

@joaomoreno
Copy link
Member

joaomoreno commented Nov 24, 2023

Fixed in 2.3.8, pipeline running

@joaomoreno
Copy link
Member

@connor4312
Copy link
Member

Thank you for looking into and fixing this! I'm surprised this didn't surface in a more explicit way. I owe you a testing/debug bug fix of your choice 😁

@joaomoreno
Copy link
Member

Yeah me too! It turns out tar -xzf also enjoys munching on zip files and just does weird things with them. 😄

alexr00 added a commit to microsoft/vscode-pull-request-github that referenced this issue Nov 28, 2023
Fixes the frequently failing pipeline
See microsoft/vscode-test#246
@alexr00
Copy link
Member

alexr00 commented Nov 28, 2023

@connor4312
Copy link
Member

Please ensure you're using a version of Node >=16 where the AbortController is available.

alexr00 added a commit to microsoft/vscode-pull-request-github that referenced this issue Nov 29, 2023
* Update test-electron package
Fixes the frequently failing pipeline
See microsoft/vscode-test#246

* udpate node in pipelines
@alexr00
Copy link
Member

alexr00 commented Nov 29, 2023

That was it, thanks!

NikolaRHristov pushed a commit to CodeEditorLand/LandTest that referenced this issue Dec 11, 2023
Fixes microsoft#246 (possibly)
though it requires microsoft/vscode-update-server#167
in order to be fully functional. At minimum it validates the length of
the stream matches the content-length the CDN told us about, and it
will validate SHA256 checksums when the update server provides them.

(I looked at getting them other ways, but it's very roundabout; the
update server should just add them on all its headers.)
NikolaRHristov pushed a commit to CodeEditorLand/LandTest that referenced this issue Dec 11, 2023
shenglol added a commit to Azure/bicep that referenced this issue Dec 13, 2023
The extension e2e tests are failing due to
microsoft/vscode-test#246. Update
@vscode/test-electron to fix it.
strager added a commit to quick-lint/quick-lint-js that referenced this issue Feb 17, 2024
'yarn test' downloads Visual Studio Code into a temporary directory. A
change in Visual Studio Code's server caused the download to break [1].
Example error message on Windows:

    Test error: Error: spawn
    P:\quick-lint-js\plugin\vscode\.vscode-test\vscode-win32-x64-archive-1.55.2\Code.exe ENOENT
    Exit code:   -4058

Fix the broken downloading by applying a patch to the download
script [2].

[1] microsoft/vscode-test#246
[2] microsoft/vscode-test@62881f0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issue identified by VS Code Team member as probable bug
Projects
None yet
4 participants