Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vcpkg export requires packages directory #9781

Closed
katusk opened this issue Jan 21, 2020 · 1 comment · Fixed by #11015
Closed

vcpkg export requires packages directory #9781

katusk opened this issue Jan 21, 2020 · 1 comment · Fixed by #11015
Assignees
Labels
category:vcpkg-bug The issue is with the vcpkg system (including helper scripts in `scripts/cmake/`)

Comments

@katusk
Copy link

katusk commented Jan 21, 2020

Describe the bug
The vcpkg export command fails if the packages directory (under the vcpkg root directory) does not contain the to-be-exported package. If you vcpkg install a package with the --clean-after-build switch, the packages directory is not going to be present after a successful install.

Environment

  • OS: Windows 10
  • Compiler: Microsoft Visual Studio Professional 2017 Version 15.9.16

To Reproduce
Using the flatbuffers library as an example, you can use any other libraries if you want, the results should be the same. Example steps to reproduce the behavior with a x64-windows triplet (the triplet itself is irrelevant):

  1. .\vcpkg remove flatbuffers:x64-windows
  2. .\vcpkg install flatbuffers:x64-windows --clean-after-build
  3. .\vcpkg export flatbuffers:x64-windows --raw --output="vcpkg-export-flatbuffers"
  4. See error Source directory C:/workspace/vcpkg/packages/flatbuffers_x64-windows does not exist (assuming your vcpkg root is at C:/workspace/vcpkg).

Results are the same with any of the other export types: "--nuget --ifw --zip --7zip".

Expected behavior
The vcpkg export command in the above should succeed even if the packages directory is not present in the vcpkg root directory. If you omit the --clean-after-build switch (packages directory remains intact), the export command succeeds:

  1. .\vcpkg remove flatbuffers:x64-windows
  2. .\vcpkg install flatbuffers:x64-windows
  3. .\vcpkg export flatbuffers:x64-windows --raw --output="vcpkg-export-flatbuffers"

Failure logs

c:\workspace\vcpkg>vcpkg export flatbuffers:x64-windows --raw --output="vcpkg-export-flatbuffers"
The following packages are already built and will be exported:
    flatbuffers:x64-windows
Exporting package flatbuffers:x64-windows...
Source directory C:/workspace/vcpkg/packages/flatbuffers_x64-windows does not exist

Additional context
I do not think that the packages directory should be necessary to perform a successful export operation. The installed directory in the vcpkg root directory has all the necessary information (assuming x64-windows triplet for the sake of an example):

  • The installed/x64-windows directory has all the installed binaries and headers at one place.
  • The *_x64-windows.list files under installed/vcpkg/info have all the information describing what files in the installed/x64-windows directory belong to a given library.

E.g. on my machine right now, the installed/vcpkg/info/flatbuffers_1.11.0-1_x64-windows.list file contains the following:

x64-windows/
x64-windows/debug/
x64-windows/debug/lib/
x64-windows/debug/lib/flatbuffers.lib
x64-windows/include/
x64-windows/include/flatbuffers/
x64-windows/include/flatbuffers/base.h
x64-windows/include/flatbuffers/code_generators.h
x64-windows/include/flatbuffers/flatbuffers.h
x64-windows/include/flatbuffers/flatc.h
x64-windows/include/flatbuffers/flexbuffers.h
x64-windows/include/flatbuffers/grpc.h
x64-windows/include/flatbuffers/hash.h
x64-windows/include/flatbuffers/idl.h
x64-windows/include/flatbuffers/minireflect.h
x64-windows/include/flatbuffers/reflection.h
x64-windows/include/flatbuffers/reflection_generated.h
x64-windows/include/flatbuffers/registry.h
x64-windows/include/flatbuffers/stl_emulation.h
x64-windows/include/flatbuffers/util.h
x64-windows/lib/
x64-windows/lib/flatbuffers.lib
x64-windows/share/
x64-windows/share/flatbuffers/
x64-windows/share/flatbuffers/FlatbuffersConfig.cmake
x64-windows/share/flatbuffers/FlatbuffersConfigVersion.cmake
x64-windows/share/flatbuffers/FlatbuffersTargets-debug.cmake
x64-windows/share/flatbuffers/FlatbuffersTargets-release.cmake
x64-windows/share/flatbuffers/FlatbuffersTargets.cmake
x64-windows/share/flatbuffers/FlatcTargets-debug.cmake
x64-windows/share/flatbuffers/FlatcTargets-release.cmake
x64-windows/share/flatbuffers/FlatcTargets.cmake
x64-windows/share/flatbuffers/copyright
x64-windows/share/flatbuffers/vcpkg_abi_info.txt
x64-windows/tools/
x64-windows/tools/flatbuffers/
x64-windows/tools/flatbuffers/flatc.exe
@katusk
Copy link
Author

katusk commented Jan 21, 2020

Hmm, both

call Install::install_files_and_write_listfile (searched for "Exporting package"), that does what its name suggests, and checks for the source directory (searched for "Source directory"; this is substituted for the appropriate subdirectory under packages if I am not mistaken) at

I think we should call something like read_listfile_and_export_files (just made this name up) at this point and skip checks that are related to the packages directory.

In my mind, at least, the natural sequence of steps for a vcpkg install (points 1-4) and then a vcpkg export (point 5) are the following:

  1. Download the package (downloads directory)
  2. Build the package (buildtrees directory)
  3. Gather build artifacts for install (packages directory)
  4. Install the package (installed directory)
  5. Export the package (custom export directory)

...where every step should only rely on files from a previous step, I think. And I also consider the directories in points 1-3 to be temporary that could be deleted at any time based on the behaviour and help description of the --clean-after-build switch.

Please let me know what is the best way to proceed, and I am happy to send a PR with the necessary changes.

ras0219 pushed a commit to ras0219/vcpkg that referenced this issue Apr 24, 2020
ras0219-msft added a commit that referenced this issue Jun 17, 2020
…1015)

* [vcpkg] Fix issue #9781 by exporting from the installed directory

* [vcpkg] Address code review comments for #11015

* [vcpkg] Remove duplicate triplet in installed path from exports

Co-authored-by: Robert Schumacher <[email protected]>
@JackBoosY JackBoosY added the category:vcpkg-bug The issue is with the vcpkg system (including helper scripts in `scripts/cmake/`) label Jun 18, 2020
JangBoo pushed a commit to JangBoo/vcpkg that referenced this issue Jun 18, 2020
…ctory (microsoft#11015)

* [vcpkg] Fix issue microsoft#9781 by exporting from the installed directory

* [vcpkg] Address code review comments for microsoft#11015

* [vcpkg] Remove duplicate triplet in installed path from exports

Co-authored-by: Robert Schumacher <[email protected]>
strega-nil pushed a commit to strega-nil/vcpkg that referenced this issue May 5, 2021
…ctory (microsoft#11015)

* [vcpkg] Fix issue microsoft#9781 by exporting from the installed directory

* [vcpkg] Address code review comments for microsoft#11015

* [vcpkg] Remove duplicate triplet in installed path from exports

Co-authored-by: Robert Schumacher <[email protected]>
kLiHz added a commit to kLiHz/vcpkg-build-opencv that referenced this issue Apr 17, 2023
As in issue microsoft/vcpkg#9781
and merged pull request microsoft/vcpkg#11015
, vcpkg doesn't use `packages` dir for either installing or exporting.

Since now disabling binary cache, directory `installed` should be used.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:vcpkg-bug The issue is with the vcpkg system (including helper scripts in `scripts/cmake/`)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants