Page MenuHomePhabricator

[compiler-rt] Build custom libcxx with libcxxabi
ClosedPublic

Authored by Hahnfeld on Feb 10 2019, 3:58 AM.

Details

Summary

This changes add_custom_libcxx to also build libcxxabi and merges
the two into a static and hermetic library.
There are multiple advantages:

  1. The resulting libFuzzer doesn't expose C++ internals and looks like a plain C library.
  2. We don't have to manually link in libstdc++ to provide cxxabi.
  3. The sanitizer tests cannot interfere with an installed version of libc++.so in LD_LIBRARY_PATH.

Diff Detail

Event Timeline

Hahnfeld created this revision.Feb 10 2019, 3:58 AM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptFeb 10 2019, 3:58 AM
Herald added subscribers: llvm-commits, Restricted Project, christof and 2 others. · View Herald Transcript
phosek added inline comments.Feb 10 2019, 5:35 PM
compiler-rt/lib/fuzzer/CMakeLists.txt
138 ↗(On Diff #186147)

I don't think this is correct either. The problem is that on Linux you might end up mixing different C++ libraries and different C++ ABI implementations. Ideally, libFuzzer would be built against hermetic libc++abi and libc++ so it can be linked into any program independently of which C++ library it uses (or even plain C programs). This wasn't possible before, but it's possible now that libunwind, libc++abi and libc++ supports building hermetic static library.

So the ideal solution here is to build libc++abi in addition to libc++ (as hermetic libraries) and use those the same way we already do today. The resulting libFuzzer library would look like a C library from the consumer perspective. I was planning on doing that, I just haven't had time yet but I hope to get to it soon.

Hahnfeld updated this revision to Diff 186705.Feb 13 2019, 11:19 AM
Hahnfeld retitled this revision from [compiler-rt] Introduce SANITIZER_LIBCXX_CXX_ABI to [compiler-rt] Build custom libcxx with libcxxabi.
Hahnfeld edited the summary of this revision. (Show Details)

@phosek Excellent idea. I tried to implement what you described and it seems to work great! The only minor disadvantage is that we now need to disable the tests if COMPILER_RT_LIBCXXABI_PATH is not found...

@phosek Excellent idea. I tried to implement what you described and it seems to work great! The only minor disadvantage is that we now need to disable the tests if COMPILER_RT_LIBCXXABI_PATH is not found...

This is awesome, exactly what I had in mind, thank you so much for working on this!

compiler-rt/cmake/Modules/CustomLibcxx/CMakeLists.txt
21 ↗(On Diff #186705)

Should we also set -DLIBCXX_ENABLE_EXCEPTIONS=OFF to avoid the dependency on unwinder (e.g. libunwind or libgcc_s)? We already do this for libFuzzer but would it also make sense for MSan and TSan copies?

Hahnfeld updated this revision to Diff 186803.Feb 14 2019, 1:40 AM
Hahnfeld marked 3 inline comments as done.

Cleaunup CustomLibcxx/CMakeLists.txt and build custom libcxx without exceptions.

Hahnfeld marked an inline comment as done.Feb 14 2019, 1:43 AM
Hahnfeld added inline comments.
compiler-rt/cmake/Modules/CustomLibcxx/CMakeLists.txt
21 ↗(On Diff #186705)

Yes, I guess that makes sense.

compiler-rt/lib/fuzzer/CMakeLists.txt
139 ↗(On Diff #186803)

I've also tried to remove the ABI namespace (and adjusting LIBFUZZER_CFLAGS) but that resulted in Segfaults during runtime. From a first look it appeared that some relocations were broken in the binary, resulting in calling address 0x0, but I'll need to investigate. I'd say that's out of the scope of this patch anyway.

phosek accepted this revision.Feb 14 2019, 8:50 PM

LGTM

This revision is now accepted and ready to land.Feb 14 2019, 8:50 PM
This revision was automatically updated to reflect the committed changes.

I needed to push rCRT354231 to fix the sanitizer bots. Let me know if this change is appropriate or want me to revert. In that case we need to run another round of clobber builds to fix CMake configuration (needed because the patch changes the source directory for ExternalProject_Add).

This also broke our toolchain builders. The relevant part of the log is:

[357/763] Performing configure step for 'libcxx_fuzzer_aarch64'
-- The C compiler identification is Clang 9.0.0
-- The CXX compiler identification is Clang 9.0.0
-- Check for working C compiler: /b/s/w/ir/kitchen-workdir/recipe_cleanup/clangrCZ026/llvm_build_dir/./bin/clang
-- Check for working C compiler: /b/s/w/ir/kitchen-workdir/recipe_cleanup/clangrCZ026/llvm_build_dir/./bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - failed
-- Detecting CXX compile features
-- Detecting CXX compile features - failed
-- Configuring for standalone build.
-- Linker detection: LLD
-- Performing Test LLVM_LIBSTDCXX_MIN
-- Performing Test LLVM_LIBSTDCXX_MIN - Failed
CMake Error at /b/s/w/ir/kitchen-workdir/llvm-project/llvm/cmake/modules/CheckCompilerVersion.cmake:72 (message):
  libstdc++ version must be at least 4.8.
Call Stack (most recent call first):
  /b/s/w/ir/kitchen-workdir/llvm-project/llvm/cmake/modules/HandleLLVMOptions.cmake:9 (include)
  /b/s/w/ir/kitchen-workdir/llvm-project/libcxxabi/cmake/Modules/HandleOutOfTreeLLVM.cmake:99 (include)
  /b/s/w/ir/kitchen-workdir/llvm-project/libcxxabi/cmake/Modules/HandleOutOfTreeLLVM.cmake:142 (configure_out_of_tree_llvm)
  /b/s/w/ir/kitchen-workdir/llvm-project/libcxxabi/CMakeLists.txt:29 (include)


-- Configuring incomplete, errors occurred!

I don't know yet why the LLVM_LIBSTDCXX_MIN check is failing but I'm going to do a local build to find out.

I found the issue, but I don't know how to easily solve this. The problem is that when being built in standalone mode, libc++abi and libc++ includes LLVM's HandleLLVMOptions which uses checks that try to link C++ library, but that may not be ready yet in case of the runtimes build. In runtimes build I solve this by [adding -nostdlib++ to required flags](https://github.com/llvm/llvm-project/blob/master/llvm/runtimes/CMakeLists.txt#L100). I can do that because in case of runtimes build I know that we'll be always using the just built Clang that supports that flag. I could address this issue by also including -nostdlib++ in required flags in compiler-rt/cmake/Modules/CustomLibcxx/CMakeLists.txt but that's not a general solution because that file in theory also supports other compilers that may not have this flag (older Clang versions, GCC). We could try and use -nostdlib and then include builtins manually but that's a lot of extra code that would need to be duplicated to handle cases like compiler-rt builtins and libgcc.

The general solution is to stop relying on checks that assume working C++ library from runtimes (it's where we're building C++ library after all, we should assume a working C++ library), but that's going to require more significant refactor and I'd like to unbreak our bots ASAP so we'll need some other solution.

I found the issue, but I don't know how to easily solve this. The problem is that when being built in standalone mode, libc++abi and libc++ includes LLVM's HandleLLVMOptions which uses checks that try to link C++ library, but that may not be ready yet in case of the runtimes build. In runtimes build I solve this by [adding -nostdlib++ to required flags](https://github.com/llvm/llvm-project/blob/master/llvm/runtimes/CMakeLists.txt#L100). I can do that because in case of runtimes build I know that we'll be always using the just built Clang that supports that flag. I could address this issue by also including -nostdlib++ in required flags in compiler-rt/cmake/Modules/CustomLibcxx/CMakeLists.txt but that's not a general solution because that file in theory also supports other compilers that may not have this flag (older Clang versions, GCC). We could try and use -nostdlib and then include builtins manually but that's a lot of extra code that would need to be duplicated to handle cases like compiler-rt builtins and libgcc.

The general solution is to stop relying on checks that assume working C++ library from runtimes (it's where we're building C++ library after all, we should assume a working C++ library), but that's going to require more significant refactor and I'd like to unbreak our bots ASAP so we'll need some other solution.

D58331 is what I came up with so far.

@phosek Sorry for the breakage. I guess there are just too many configurations for non-trivial changes to the build system :-/