[CMake] Enable LLVM_ENABLE_PER_TARGET_RUNTIME_DIR by default on *BSD

[MaskRay]

Summary

Similar to D107799 but for *BSD (DragonFlyBSD, FreeBSD, NetBSD, OpenBSD, etc).
This Linux default has been in main and release/15.x for a while.

CMAKE_SYSTEM_PROCESSOR MATCHES "^arm" is excluded for now. It is unclear
whether the Linux arm vs armhf issue applies to *BSD.

Link: https://discourse.llvm.org/t/rfc-time-to-drop-legacy-runtime-paths/64628

[arichardson]

I think this should generally be safe but I have run into issues where the 64-bit sanitizer libraries were being linked with -m32, since the --target value is used for the clang runtime libdir instead of the adjusted triple.

[MaskRay]

In D110126#3011397, @arichardson wrote:

I think this should generally be safe but I have run into issues where the 64-bit sanitizer libraries were being linked with -m32, since the --target value is used for the clang runtime libdir instead of the adjusted triple.

-m32 works. For example, on my FreeBSD amd64, -m32 picks /usr/home/ray/llvm/Release/lib/clang/14.0.0/lib/i386-unknown-freebsd12.1/libclang_rt.asan.a

[arichardson]

In D110126#3013056, @MaskRay wrote:

In D110126#3011397, @arichardson wrote:

I think this should generally be safe but I have run into issues where the 64-bit sanitizer libraries were being linked with -m32, since the --target value is used for the clang runtime libdir instead of the adjusted triple.

-m32 works. For example, on my FreeBSD amd64, -m32 picks /usr/home/ray/llvm/Release/lib/clang/14.0.0/lib/i386-unknown-freebsd12.1/libclang_rt.asan.a

Thanks for checking, the last time I tried I ended up posting https://reviews.llvm.org/D100054, but if that is no longer necessary I can abandon it instead of keeping it on my (much too long) list of things I need to get back to.

[dim]

I tried building head-of-main with LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=ON, and one thing I noticed is that the directories have the minor version swizzled in:

% ls -l /share/dim/llvm/llvmorg-14-init-4370-gd87d1aa07612-freebsd14-amd64-ninja-clang-rel-1/lib/clang/14.0.0/lib
total 8
drwxr-xr-x  2 dim  dim  1024 2021-09-21 11:07:47 i386-unknown-freebsd14.0/
drwxr-xr-x  2 dim  dim  1536 2021-09-21 11:07:47 x86_64-unknown-freebsd14.0/

I'd rather not have that, since we've had other trouble with it, see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258377 (where Python builds get confused by the dot in the -print-multiarch output).

@emaste @arichardson do you agree with that? I think the best solution is to use only the major version number, e.g. i386-unknown-freebsd14 and x86_64-unknown-freebsd14? But this may require some changes in the code that handles triples.

(Note that Linux appears to get rid of the OS version entirely, e.g. it has lib/clang/14.0.0/lib/x86_64-unknown-linux-gnu/.

[MaskRay]

The default target triple (clang -dumpmachine) is decided by config.guess (which will be removed by D109837).

If you prefer a target triple without a suffix, just specify -DLLVM_HOST_TRIPLE=x86_64-pc-freebsd (or unknown) explicitly for newer clang packages.
For pc, you'll then get lib/clang/14.0.0/lib/i386-pc-freebsd/libclang_rt.asan.so lib/clang/14.0.0/lib/x86_64-pc-freebsd/libclang_rt.asan.so

[MaskRay]

Ping :)

[MaskRay]

@joerg @krytarowski Hope that NetBSD can switch to the layout as well.

[MaskRay]

Enable for all *BSD

[dim]

I'm fine with it, adding @brooks so he's aware for the FreeBSD llvm port.

[jrtc27]

What is the arm vs armhf issue? FreeBSD uses arm-(unknown-)freebsd-gnueabi and -gnueabihf depending on the float ABI so the triples are different, if that's the question, though on RISC-V both riscv64 and riscv64sf exist but use the same triple

[MaskRay]

In D110126#3737497, @jrtc27 wrote:

What is the arm vs armhf issue? FreeBSD uses arm-(unknown-)freebsd-gnueabi and -gnueabihf depending on the float ABI so the triples are different, if that's the question, though on RISC-V both riscv64 and riscv64sf exist but use the same triple

https://lab.llvm.org/buildbot/#/builders/178 (clang-armv8-lld-2stage, -mcpu=cortex-a57, -DLLVM_ENABLE_PROJECTS=compiler-rt;llvm;clang-tools-extra;lld;clang) used to fail with LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=on.
It might be due to the subarch part "v8l". I don't know why it failed.

-- LLVM host triple: armv8l-unknown-linux-gnueabihf
-- LLVM default target triple: armv8l-unknown-linux-gnueabihf

rg 'arm.*freebsd' clang/test gives me Frontend/gnu-mcount.c (armv7-unknown-freebsd-gnueabihf). If a BSD doesn't use subarch for arm, I think it will likely work without any change. If a BSD uses arch for arm, there may be some risk. It'd be nice if someone can investigate the issue.

AIUI -LLVM_ENABLE_PER_TARGET_RUNTIME_DIR=off -DLLVM_ENABLE_PROJECTS=compiler-rt uses the layout which is to be phased out (see https://discourse.llvm.org/t/rfc-time-to-drop-legacy-runtime-paths/64628).

[jrtc27]

Ah right yes we do put armv6 and armv7 in the triple

[brooks]

Seems fine to me. There will likely be some bumps the FreeBSD port, but should be manageable, especially if it's coming in 16.

[MaskRay]

Ping:)

[MaskRay]

@dim @emaste Ping

[dim]

Yes, this should probably be fine for 16.

llvm/CMakeLists.txt

if(CMAKE_HOST_APPLE AND APPLE)
   include(UseLibtool)
 endif()
 
 # Override the default target with an environment variable named by LLVM_TARGET_TRIPLE_ENV.
 set(LLVM_TARGET_TRIPLE_ENV CACHE STRING "The name of environment variable to override default target. Disabled by blank.")
 mark_as_advanced(LLVM_TARGET_TRIPLE_ENV)
 
 # Per target dir not yet supported on Arm 32 bit due to arm vs armhf handling
-if(CMAKE_SYSTEM_NAME MATCHES "Linux" AND NOT CMAKE_SYSTEM_PROCESSOR MATCHES "^arm")
+if(CMAKE_SYSTEM_NAME MATCHES "BSD|Linux" AND NOT CMAKE_SYSTEM_PROCESSOR MATCHES "^arm")
   set(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default ON)
 else()
   set(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default OFF)
 endif()
 set(LLVM_ENABLE_PER_TARGET_RUNTIME_DIR ${LLVM_ENABLE_PER_TARGET_RUNTIME_DIR_default} CACHE BOOL
   "Enable per-target runtimes directory")
 
 set(LLVM_PROFDATA_FILE "" CACHE FILEPATH