On FreeBSD, add -pthread to ASan dynamic compile flags for tests

[dim]

Summary

Otherwise, lots of these tests fail with a CHECK error similar to:

12345==AddressSanitizer CHECK failed: compiler-rt/lib/asan/asan_posix.cpp:120 "((0)) == ((pthread_key_create(&tsd_key, destructor)))" (0x0, 0x4e)

This is because the default pthread stubs in FreeBSD's libc always
return failures (such as ENOSYS for pthread_key_create) in case the
pthread library is not linked in.

[dim]

Note that as of rG1db4318766256f25a03ef80af8dbb3f99743ebe9, applying this change results in the Passed testcases increasing from 49259 to 49666, so more than 400 Failed test cases are fixed by it.

[arichardson]

Can we instead add a dependency on libpthread.so/libthr.so to libasan.so?

[arichardson]

In D85082#2190291, @arichardson wrote:

Can we instead add a dependency on libpthread.so/libthr.so to libasan.so?

Never mind, it seems like it's already there:

File: ./lib/clang/12.0.0/lib/freebsd/libclang_rt.asan-x86_64.so
Format: elf64-x86-64
Arch: x86_64
AddressSize: 64bit
LoadName: libclang_rt.asan-x86_64.so
NeededLibraries [
  libc++.so.1
  libc.so.7
  libcxxrt.so.1
  libdl.so.1
  libgcc_s.so.1
  libm.so.5
  librt.so.1
  libthr.so.3
]

I thought libthr functions should be picked up correctly (at least after https://reviews.freebsd.org/D21088)

[dim]

In D85082#2190303, @arichardson wrote:

In D85082#2190291, @arichardson wrote:

Can we instead add a dependency on libpthread.so/libthr.so to libasan.so?

Never mind, it seems like it's already there:

File: ./lib/clang/12.0.0/lib/freebsd/libclang_rt.asan-x86_64.so
Format: elf64-x86-64
Arch: x86_64
AddressSize: 64bit
LoadName: libclang_rt.asan-x86_64.so
NeededLibraries [
  libc++.so.1
  libc.so.7
  libcxxrt.so.1
  libdl.so.1
  libgcc_s.so.1
  libm.so.5
  librt.so.1
  libthr.so.3
]

Indeed. However, on FreeBSD we don't call __asan_init via .preinit_array (see https://github.com/llvm/llvm-project/blob/master/compiler-rt/lib/sanitizer_common/sanitizer_internal_defs.h#L101), and so ASan gets initialized only when some function like malloc calls into ASan, e.g.:

#0  __asan::AsanTSDInit (destructor=0x8002a0430 <__asan::PlatformTSDDtor(void*)>) at /usr/src/contrib/llvm-project/compiler-rt/lib/asan/asan_posix.cpp:89
#1  0x000000080029ef39 in __asan::AsanInitInternal () at /usr/src/contrib/llvm-project/compiler-rt/lib/asan/asan_rtl.cpp:465
#2  0x000000080029edd7 in __asan::AsanInitFromRtl () at /usr/src/contrib/llvm-project/compiler-rt/lib/asan/asan_rtl.cpp:537
#3  0x00000008002cb66e in __interceptor_readlink (path=0x800c73203 "/etc/malloc.conf", buf=0x7fffffffd0b0 "\336\003", bufsiz=1024) at /usr/src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:7009
#4  0x0000000800d7b106 in obtain_malloc_conf (which_source=2, buf=0x7fffffffd0b0 "\336\003") at jemalloc_jemalloc.c:984
#5  malloc_conf_init_helper (sc_data=0x0, bin_shard_sizes=0x0, initial_call=true, opts_cache=0x7fffffffd4c0, buf=<optimized out>) at jemalloc_jemalloc.c:1042
#6  0x0000000800d7ab81 in malloc_conf_init (sc_data=0x7fffffffb700, bin_shard_sizes=0x7fffffffb670) at jemalloc_jemalloc.c:1449
#7  malloc_init_hard_a0_locked () at jemalloc_jemalloc.c:1509
#8  0x0000000800d78814 in malloc_init_hard () at jemalloc_jemalloc.c:1754
#9  malloc_init () at jemalloc_jemalloc.c:227
#10 0x000000080020cf4d in objlist_call_init (list=<optimized out>, lockstate=<optimized out>) at /usr/src/libexec/rtld-elf/rtld.c:2867
#11 0x000000080020bc1b in _rtld (sp=0x7fffffffe550, exit_proc=0x7fffffffe520, objp=0x7fffffffe528) at /usr/src/libexec/rtld-elf/rtld.c:847
#12 0x0000000800209419 in rtld_start () at /usr/src/libexec/rtld-elf/amd64/rtld_start.S:39
#13 0x0000000000000000 in ?? ()

In an executable, linked dynamically to libasan.so, at this point pthread_key_create() still refers to the stubbed libc function, *iff* the executable itself is not directly linked against the threading library, libthr.so. This can be fixed (or rather worked around) by adding -pthread, as I am proposing.

It also means that *any* executable wanting to make use of dynamic libasan.so should be linked with -pthread. But I think the number of actual users approaches zero, except for the compiler-rt unit tests.

The true solution would be to fold FreeBSD libthr functionality into libc, but I have understood that this is a big piece of technical debt which has been dragged around for years now, so it is not likely to be solved very soon, and certainly not for all existing releases of FreeBSD out there.

I thought libthr functions should be picked up correctly (at least after https://reviews.freebsd.org/D21088)

Nope, this has never worked, and does not work on very recent FreeBSD -CURRENT either.

[arichardson]

This patch does not appear to make any difference for me when using check-asan on FreeBSD 12.1-RELEASE-p2.
I'm seeing the following two failures for check-asan either way:

Failed Tests (2):

AddressSanitizer-i386-freebsd :: TestCases/Posix/fread_fwrite.cpp
AddressSanitizer-x86_64-freebsd :: TestCases/Posix/high-address-dereference.c

But if it helps on other versions, LGTM.

compiler-rt/test/asan/lit.cfg.py

clang_asan_static_cflags = (["-fsanitize=address",
                             config.debug_info_flags + target_cflags)
 if config.target_arch == 's390x':
   clang_asan_static_cflags.append("-mbackchain")
 clang_asan_static_cxxflags = config.cxx_mode_flags + clang_asan_static_cflags
 
 asan_dynamic_flags = []
 if config.asan_dynamic:
   asan_dynamic_flags = ["-shared-libasan"]
-  # On Windows, we need to simulate "clang-cl /MD" on the clang driver side.
   if platform.system() == 'Windows':
+    # On Windows, we need to simulate "clang-cl /MD" on the clang driver side.
     asan_dynamic_flags += ["-D_MT", "-D_DLL", "-Wl,-nodefaultlib:libcmt,-defaultlib:msvcrt,-defaultlib:oldnames"]
+  elif platform.system() == 'FreeBSD':
+    # On FreeBSD, we need to add -pthread to ensure pthread functions are available.
+    asan_dynamic_flags += ['-pthread']
   config.available_features.add("asan-dynamic-runtime")
 else:
   config.available_features.add("asan-static-runtime")
 clang_asan_cflags = clang_asan_static_cflags + asan_dynamic_flags
 clang_asan_cxxflags = clang_asan_static_cxxflags + asan_dynamic_flags
 
 # Add win32-(static|dynamic)-asan features to mark tests as passing or failing
 # in those modes. lit doesn't support logical feature test combinations.