Page MenuHomePhabricator

[CMake] Avoid accidental C++ standard library dependency in sanitizers
AcceptedPublic

Authored by phosek on Tue, Oct 6, 1:26 PM.

Details

Summary

While sanitizers don't use C++ standard library, we could still end
up accidentally including or linking it just by the virtue of using
the C++ compiler. Pass -nostdinc++ and -nostdlib++ to avoid these
accidental dependencies.

Diff Detail

Unit TestsFailed

TimeTest
360 mswindows > LLVM.tools/llvm-cov::warnings.h
Script: -- : 'RUN: at line 1'; c:\ws\w64\llvm-project\premerge-checks\build\bin\llvm-cov.exe show C:\ws\w64\llvm-project\premerge-checks\llvm\test\tools\llvm-cov/Inputs/prevent_false_instantiations.covmapping -instr-profile C:\ws\w64\llvm-project\premerge-checks\llvm\test\tools\llvm-cov/Inputs/elf_binary_comdat.profdata -path-equivalence=/tmp,C:\ws\w64\llvm-project\premerge-checks\llvm\test\tools\llvm-cov | c:\ws\w64\llvm-project\premerge-checks\build\bin\filecheck.exe C:\ws\w64\llvm-project\premerge-checks\llvm\test\tools\llvm-cov\warnings.h -allow-empty -check-prefix=FAKE-FILE-STDOUT

Event Timeline

phosek created this revision.Tue, Oct 6, 1:26 PM
Herald added a project: Restricted Project. · View Herald TranscriptTue, Oct 6, 1:26 PM
Herald added subscribers: Restricted Project, mgorny. · View Herald Transcript
phosek requested review of this revision.Tue, Oct 6, 1:26 PM

LGTM, but I'll give the sanitizer folks a bit of time to review as well.

smeenai accepted this revision.Wed, Oct 14, 6:15 PM

This is needed to avoid potential race conditions where some of the C++ headers have been copied but not all of them, and some other runtime that doesn't depend on the C++ headers sees this incomplete state and gets a spurious build failure, so LGTM.

This revision is now accepted and ready to land.Wed, Oct 14, 6:15 PM
This revision was landed with ongoing or failed builds.Wed, Oct 14, 6:27 PM
This revision was automatically updated to reflect the committed changes.
vitalybuka added a comment.EditedWed, Oct 14, 6:28 PM

ninja check-tsan

1 error generated.
[18/23] Generating TsanUnitTestsObjects.tsan_clock_test.cpp.x86_64.o
FAILED: projects/compiler-rt/lib/tsan/tests/unit/TsanUnitTestsObjects.tsan_clock_test.cpp.x86_64.o 
cd /usr/local/google/home/vitalybuka/src/llvm.git/out/z/projects/compiler-rt/lib/tsan/tests/unit && /usr/local/google/home/vitalybuka/src/llvm.git/out/z/./bin/clang -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -nostdinc++ -fPIE -fno-rtti -Wno-covered-switch-default -DGTEST_NO_LLVM_SUPPORT=1 -DGTEST_HAS_RTTI=0 -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/llvm/utils/unittest/googletest/include -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/llvm/utils/unittest/googletest -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/include -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/tsan/rtl -DGTEST_HAS_RTTI=0 -m64 -c -o TsanUnitTestsObjects.tsan_clock_test.cpp.x86_64.o /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/tsan/tests/unit/tsan_clock_test.cpp
In file included from /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/tsan/tests/unit/tsan_clock_test.cpp:14:
/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/llvm/utils/unittest/googletest/include/gtest/gtest.h:54:10: fatal error: 'limits' file not found
#include <limits>
         ^~~~~~~~
1 error generated.
phosek reopened this revision.Wed, Oct 14, 6:46 PM

ninja check-tsan

1 error generated.
[18/23] Generating TsanUnitTestsObjects.tsan_clock_test.cpp.x86_64.o
FAILED: projects/compiler-rt/lib/tsan/tests/unit/TsanUnitTestsObjects.tsan_clock_test.cpp.x86_64.o 
cd /usr/local/google/home/vitalybuka/src/llvm.git/out/z/projects/compiler-rt/lib/tsan/tests/unit && /usr/local/google/home/vitalybuka/src/llvm.git/out/z/./bin/clang -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -nostdinc++ -fPIE -fno-rtti -Wno-covered-switch-default -DGTEST_NO_LLVM_SUPPORT=1 -DGTEST_HAS_RTTI=0 -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/llvm/utils/unittest/googletest/include -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/llvm/utils/unittest/googletest -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/include -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/tsan/rtl -DGTEST_HAS_RTTI=0 -m64 -c -o TsanUnitTestsObjects.tsan_clock_test.cpp.x86_64.o /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/tsan/tests/unit/tsan_clock_test.cpp
In file included from /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/tsan/tests/unit/tsan_clock_test.cpp:14:
/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/llvm/utils/unittest/googletest/include/gtest/gtest.h:54:10: fatal error: 'limits' file not found
#include <limits>
         ^~~~~~~~
1 error generated.

Reverted, I'll update the change to avoid using -nostdinc++ for tests.

This revision is now accepted and ready to land.Wed, Oct 14, 6:46 PM

Also android bot is likely will complain. You can reproduce according to:
https://github.com/google/sanitizers/wiki/SanitizerBotReproduceBuild
BUILDBOT_REVISION= BUILDBOT_MONO_REPO_PATH=<my llvm checkout path>/llvm-project BUILDBOT_CLOBBER= zorg/zorg/buildbot/builders/sanitizers/buildbot_android.sh

Or just ping me and I try myself.

phosek updated this revision to Diff 298308.Wed, Oct 14, 11:14 PM

Also android bot is likely will complain. You can reproduce according to:
https://github.com/google/sanitizers/wiki/SanitizerBotReproduceBuild
BUILDBOT_REVISION= BUILDBOT_MONO_REPO_PATH=<my llvm checkout path>/llvm-project BUILDBOT_CLOBBER= zorg/zorg/buildbot/builders/sanitizers/buildbot_android.sh

Or just ping me and I try myself.

@vitalybuka I tried it locally but that script failed with CMake Error: The source directory "/src/clang-llvm/llvm_build_android_i686/llvm-project/llvm" does not exist. error, would it be possible to run it on your side?

smeenai added inline comments.Wed, Oct 14, 11:55 PM
compiler-rt/lib/tsan/tests/CMakeLists.txt
17

Won't this also need the cxx-headers dep?

Also android bot is likely will complain. You can reproduce according to:
https://github.com/google/sanitizers/wiki/SanitizerBotReproduceBuild
BUILDBOT_REVISION= BUILDBOT_MONO_REPO_PATH=<my llvm checkout path>/llvm-project BUILDBOT_CLOBBER= zorg/zorg/buildbot/builders/sanitizers/buildbot_android.sh

Or just ping me and I try myself.

@vitalybuka I tried it locally but that script failed with CMake Error: The source directory "/src/clang-llvm/llvm_build_android_i686/llvm-project/llvm" does not exist. error, would it be possible to run it on your side?

Probably earier step failed. It does not stop on errors.

I still see errors.
Probably related to compiler-rt/lib/sanitizer_common/tests/CMakeLists.txt:214

[219/440] Building CXX object lib/sanitizer_common/tests/CMakeFiles/SanitizerTest.dir/sanitizer_stacktrace_printer_test.cpp.o
FAILED: lib/sanitizer_common/tests/CMakeFiles/SanitizerTest.dir/sanitizer_stacktrace_printer_test.cpp.o 
/usr/local/google/home/vitalybuka/slow/bbot/llvm_build64/bin/clang++   -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/sanitizer_common/.. -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/sanitizer_common/tests/.. -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/sanitizer_common/tests/../.. --target=i686-linux-android --sysroot=/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686/sysroot -B/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686  -Wall -Werror -std=c++14 -Wno-unused-parameter -O3 -DNDEBUG    -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -nostdinc++ --target=i686-linux-android --sysroot=/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686/sysroot -B/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686 -Wno-covered-switch-default -Wno-suggest-override -DGTEST_NO_LLVM_SUPPORT=1 -DGTEST_HAS_RTTI=0 -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/llvm/utils/unittest/googletest/include -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/llvm/utils/unittest/googletest -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/include -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/sanitizer_common -fno-rtti -O2 -Werror=sign-compare -Wno-non-virtual-dtor -Wno-gnu-zero-variadic-macro-arguments -gline-tables-only -MD -MT lib/sanitizer_common/tests/CMakeFiles/SanitizerTest.dir/sanitizer_stacktrace_printer_test.cpp.o -MF lib/sanitizer_common/tests/CMakeFiles/SanitizerTest.dir/sanitizer_stacktrace_printer_test.cpp.o.d -o lib/sanitizer_common/tests/CMakeFiles/SanitizerTest.dir/sanitizer_stacktrace_printer_test.cpp.o -c /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/sanitizer_common/tests/sanitizer_stacktrace_printer_test.cpp
In file included from /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/sanitizer_common/tests/sanitizer_stacktrace_printer_test.cpp:14:
/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/llvm/utils/unittest/googletest/include/gtest/gtest.h:54:10: fatal error: 'limits' file not found
#include <limits>

this helped to fix sanitizer_common

diff --git a/compiler-rt/lib/sanitizer_common/tests/CMakeLists.txt b/compiler-rt/lib/sanitizer_common/tests/CMakeLists.txt
index 96c845d81cf4..d8cc089c6c25 100644
--- a/compiler-rt/lib/sanitizer_common/tests/CMakeLists.txt
+++ b/compiler-rt/lib/sanitizer_common/tests/CMakeLists.txt
@@ -202,6 +202,12 @@ if(COMPILER_RT_CAN_EXECUTE_TESTS AND NOT ANDROID)
   endforeach()
 endif()
 
+set(SANITIZER_UNITTEST_CFLAGS
+  ${SANITIZER_COMMON_CFLAGS}
+  ${SANITIZER_TEST_CFLAGS_COMMON}
+  -stdlib=libstdc++)
+list_filter(SANITIZER_UNITTEST_CFLAGS "-nostdinc++")
+
 if(ANDROID)
   foreach(arch ${SANITIZER_COMMON_SUPPORTED_ARCH})
     add_executable(SanitizerTest
@@ -211,8 +217,7 @@ if(ANDROID)
       $<TARGET_OBJECTS:RTSanitizerCommonLibc.${arch}>
       $<TARGET_OBJECTS:RTSanitizerCommonSymbolizer.${arch}>)
     set_target_compile_flags(SanitizerTest
-      ${SANITIZER_COMMON_CFLAGS}
-      ${SANITIZER_TEST_CFLAGS_COMMON})
+      ${SANITIZER_UNITTEST_CFLAGS})
     # Setup correct output directory and link flags.
     set_target_properties(SanitizerTest PROPERTIES
       RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})

then libFuzzer fails

FAILED: lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o 
/usr/local/google/home/vitalybuka/slow/bbot/llvm_build64/bin/clang++   -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/../../include --target=i686-linux-android --sysroot=/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686/sysroot -B/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686  -Wall -Werror -std=c++14 -Wno-unused-parameter -O3 -DNDEBUG    -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -Dthread_local=__thread -MD -MT lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o -MF lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o.d -o lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o -c /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerFork.cpp
In file included from /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerFork.cpp:11:
In file included from /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerCommand.h:15:
/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerDefs.h:14:10: fatal error: 'cassert' file not found
#include <cassert>
phosek updated this revision to Diff 298556.Fri, Oct 16, 12:09 AM
FAILED: lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o 
/usr/local/google/home/vitalybuka/slow/bbot/llvm_build64/bin/clang++   -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/../../include --target=i686-linux-android --sysroot=/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686/sysroot -B/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686  -Wall -Werror -std=c++14 -Wno-unused-parameter -O3 -DNDEBUG    -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -Dthread_local=__thread -MD -MT lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o -MF lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o.d -o lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o -c /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerFork.cpp
In file included from /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerFork.cpp:11:
In file included from /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerCommand.h:15:
/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerDefs.h:14:10: fatal error: 'cassert' file not found
#include <cassert>

I couldn't reproduce this failure locally. There's also no -nostdinc++ flag being used here so it's not clear to me why this is failing. Do you know if Android provides C++ standard library as part of its sysroot?

phosek marked an inline comment as done.Fri, Oct 16, 12:22 AM
FAILED: lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o 
/usr/local/google/home/vitalybuka/slow/bbot/llvm_build64/bin/clang++   -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/../../include --target=i686-linux-android --sysroot=/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686/sysroot -B/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686  -Wall -Werror -std=c++14 -Wno-unused-parameter -O3 -DNDEBUG    -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -Dthread_local=__thread -MD -MT lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o -MF lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o.d -o lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o -c /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerFork.cpp
In file included from /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerFork.cpp:11:
In file included from /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerCommand.h:15:
/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerDefs.h:14:10: fatal error: 'cassert' file not found
#include <cassert>

I couldn't reproduce this failure locally. There's also no -nostdinc++ flag being used here so it's not clear to me why this is failing. Do you know if Android provides C++ standard library as part of its sysroot?

did you try zorg script ?

FAILED: lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o 
/usr/local/google/home/vitalybuka/slow/bbot/llvm_build64/bin/clang++   -I/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/../../include --target=i686-linux-android --sysroot=/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686/sysroot -B/usr/local/google/home/vitalybuka/slow/bbot/android_ndk/standalone-i686  -Wall -Werror -std=c++14 -Wno-unused-parameter -O3 -DNDEBUG    -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -Wno-non-virtual-dtor -Dthread_local=__thread -MD -MT lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o -MF lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o.d -o lib/fuzzer/CMakeFiles/RTfuzzer.i386.dir/FuzzerFork.cpp.o -c /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerFork.cpp
In file included from /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerFork.cpp:11:
In file included from /usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerCommand.h:15:
/usr/local/google/home/vitalybuka/src/llvm.git/llvm-project/compiler-rt/lib/fuzzer/FuzzerDefs.h:14:10: fatal error: 'cassert' file not found
#include <cassert>

I couldn't reproduce this failure locally. There's also no -nostdinc++ flag being used here so it's not clear to me why this is failing. Do you know if Android provides C++ standard library as part of its sysroot?

did you try zorg script ?

Yes, I have finally managed to get it working. Turned you have to pass all paths as absolute. I think I understand what the problem is now.