Boost in yocto SDK and in .conan2

[AndersGnistrup]

What is your question?

I am using conan 2.0.8 and imports an yocto SDK containing a toolchain and some fundamental applications used in the platform.
I have made a host profile that points to the compiler. In this operation the tools.build:sysroot is set in the conf section.
This seems to generate some problems, since some libraries exists in both .conan2 and in the SDK, including the BoostConfig.cmake (among others).

During the build process the "wrong" BoostConfig.cmake is found in the SDK, but I would like to use the version in the conan2 system. If a rename the BoostConfig.cmake in the SDK the correct BoostConfig.cmake is found but this seems a "wrong" solution.

I have tried several solutions and read quite a lot of posts/docs but have not found a good solutions so what is the recommended method to import an SDK.

I have some idea's

1. Make a recipe for the SDK and and this as an requirement. This is inspired by this post.
   Distributing and using a Yocto SDK via conan #7431 (comment)
   This seems to be a valid path since stuff can be defined in components and special libraries can be masked.
2. Somehow define the CMAKE_PREFIX_PATH order or remove the path to the libraries.

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @AndersGnistrup

Thanks for reporting this.

I think we are more or less aware of this issue (#12478 for example), but it also seems complicated to fix. CMake has some limitations and doesn't have a nice mechanism to express the right priority for finding packages when there are different origins for the config.cmake scripts in the case of sysroot + using a package manager.

We need to investigate this and see if we can find any better solution.

[jcar87]

Hi @AndersGnistrup - thank you for reporting this issue.
Is there any package that you expect to be satisfied by the SDK sysroot and not by Conan? that is, a xxx-config.cmake or xxxConfig.cmake file that must resolve a call to find_package from your project.

If the answer is "NO", you can try setting CMAKE_FIND_ROOT_PATH_MODE_PACKAGE to NEVER - this way all find_package calls should be resolved by Conan, as it prevents the sysroot from being a candidate in the package search.

If the answer is "yes", it depends on whether you are able to change the find_package calls within your project.

Note that I'm assuming you are using Conan 2.0 and the CMakeToolchain and CMakeDeps generators.

[AndersGnistrup]

Hi @jcar87 - Yes, there are a few in the SDK, and yes, I am using the CMakeToolchain and CMakeDeps generators.

But, thanks for the hint about CMAKE_FIND_ROOT_PATH_MODE_PACKAGE 👍
It is only a small set of libs that is needed and these paths should be possible to set somewhere.

[jcar87]

Hi @AndersGnistrup - thanks for the follow-up.

If you know how many libraries are in both the SDK and in Conan, and it is not too many, you can try adding NO_CMAKE_FIND_ROOT_PATH to the call to find package, e.g.

find(Boost .... NO_CMAKE_FIND_ROOT_PATH)

that should have the same effect, only for those libraries. In this case, it would first look in the CMAKE_PREFIX_PATH which is set by CMakeToolchain, and should locate the Conan one - without preventing other libraries (that are not provided by Conan) to be located inside the SDK. Note that in all cases I'm specifically talking about libraries for which there exists a CMake config file inside the SDK sysroot.

[Ipiano]

I'm seeing a similar situation; one package from my dependency tree is provided by my SDK, but it's not the conan version of the package, so the cmake target names don't match what CMakeDeps tries to consume when it tries to define how use that dependency.

Rather than setting find paths and letting CMake resolve where dependencies are, is there a reason CMakeDeps/CMakeToolchain shouldn't just specify exactly the locations of dependencies Conan provides and shortcut the whole search procedure?

set(x_DIR ${CMAKE_CURRENT_LIST_DIR})
set(y_DIR ${CMAKE_CURRENT_LIST_DIR})
set(z_DIR ${CMAKE_CURRENT_LIST_DIR})

This would give a priority to things conan provides, and wouldn't mess with other search rules. When using a Yocto SDK, I prefer the "FIND_ROOT_PATH_MODE NEVER" settings that the SDK toolchain file sets, because then I don't accidentally link against my host system binaries; and right now, Conan has to change those to BOTH so that searches can find things Conan provides.

EDIT: Removed wrong and extraneous details

[AndersGnistrup]

I ended up using CMAKE_FIND_ROOT_PATH_MODE_PACKAGE = NEVER and added additional paths.

[planetmarshall]

If the answer is "NO", you can try setting CMAKE_FIND_ROOT_PATH_MODE_PACKAGE to NEVER - this way all find_package calls should be resolved by Conan, as it prevents the sysroot from being a candidate in the package search.

That's not what CMAKE_FIND_ROOT_PATH_MODE_PACKAGE does - it prevents CMAKE_FIND_ROOT_PATH being used to preffix find_package, but not the sysroot. From the CMake docs:

If set to NEVER, then the roots in CMAKE_FIND_ROOT_PATH will be ignored and only the host system root will be used

There is no value that you can set CMAKE_FIND_ROOT_PATH_MODE_PACKAGE to to prevent CMAKE_SYSROOT from being used, if it has been set. However CMake 3.23 provides CMAKE_IGNORE_PREFIX_PATH

[jwillikers]

I use the following to prevent this in the our Yocto SDK: list(PREPEND CMAKE_IGNORE_PATH ${host_sysroot}/usr/lib/cmake/Boost-1.78.0). This works when placed in the CMake toolchain provided to Conan through the user_toolchain conf variable.

[ericriff]

I'm facing exactly the same problem.

I have a yocto SDK from where I pick up a toolchain, most of the libraries my app relies on are provided by Conan2.

I expose the compiler using the PATH env variable and the sysroot through tools.build:sysroot on my host profile.

My find_package(Boost REQUIRED) calls are picking up the version provided by the yocto SDK instead of the conan one.
I managed to force CMake to use the conan2 version by just explicitly asking for the version conan provides
find_package(Boost 1.85.0 REQUIRED)
but this is very brittle as I would have to update my find_package calls if I change the boost verision. Furthermore it only works because the yocto SDK provides an older version.

[ericriff]

Another weird issue i'm tracking down and seems related:
For context, this is the folder structure of a yocto SDK

yocto-sdk
    ├── sysroots
    │   ├── armv7at2hf-neon-poky-linux-gnueabi
    │   └── x86_64-pokysdk-linux

It has a sysroot folder which contais two sub folders, the one that starts with x86 has the toolchain (compiler, linker, etc). The other one is the sysroot in the CMake/Conan sense.
I make the toolchain available by including that x86 folder on PATH and tools.build:sysroot points to armv7at2hf-neon-poky-linux-gnueabi
My builds are failing because CMake tries to use arm-poky-linux-gnueabi-ar from the armv7at2hf-neon-poky-linux-gnueabi (again, this is the sysroot so the binaries there are not for x86). I don't know why CMake prefers that binary, it is not visible on PATH.
I can replicate that by doing

find_program(RESULT arm-poky-linux-gnueabi-ar REQUIRED)
message(FATAL_ERROR ${RESULT})

...

CMake Error at CMakeLists.txt:32 (message):
  
  /opt/yocto-sdk/sysroots/armv7at2hf-neon-poky-linux-gnueabi/usr/bin/arm-poky-linux-gnueabi-ar

I don't know if there is a scenario where a find_program() call should find executables on the sysroot. All the binaries there are intended to run on target, not on the build PC.

If I delete the crosscompiled ar from the sysroot and tried again, now it finds the proper version from the x86 toolcahain

CMake Error at CMakeLists.txt:32 (message):
  
  /opt/yocto-sdk/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-gnueabi/arm-poky-linux-gnueabi-ar

[ericriff]

One last update (sorry for the noise)
Based on the comments above I managed to kind of make conan do what I want with

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE NEVER)

This is not optimal since, IIUC, then the packages from the sysroot will be completely ignored now, but I can live with that (for now?).
Is there a way to set those variables on my profile? I can hardcode them on my project's CMakeLists, but if I ever need to build a missing package then the problem will come back unless the setting are on the profile.
The old conan 1 env variables seem to not work anymore

[buildenv]
CONAN_CMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER
CONAN_CMAKE_FIND_ROOT_PATH_MODE_PACKAGE=NEVER

[planetmarshall]

I define them in a custom toolchain file and set it using the config setting: ``` tools.cmake.cmaketoolchain:user_toolchain ```

[ericriff]

I define them in a custom toolchain file and set it using the config setting: tools.cmake.cmaketoolchain:user_toolchain

I consider it, but it felt hacky. It adds a layer of indirection.

I was looking at something more conan-native, e.g.

tools.cmake.find_mode_program=never
tools.cmake.find_mode_package=never

But it looks like it is not supported.