[cmake] By default do not build compiler-rt with PGO instrumentation or use

[zhizhouy]

Summary

Currently compiler-rt doesn't officially support either PGO instrumentation or use PGO profdata to build it.

PGO related flags are passed into compiler-rt since rL372209, and causing
bugs: 45022, crbug:1018840

This patch adds several checks in compiler-rt to disable PGO related flags and provides a flag to turn on PGO for compiler-rt if needed.

[xur]

From crbug:1018840, it looks libfuzzer does not work with instrumentation. Do you know the reason.
I tried clang PGO bootstrap that enables PGO instrumentation in compiler-rt for stage2 compiler. Everything works fine -- i.e. profile runtime functions in compiel-rt generate a correct profile.

Can we put this change under an option? I can image someone might want the profile for the compiler-rt runtime.

[zhizhouy]

From crbug:1018840, it looks libfuzzer does not work with instrumentation. Do you know the reason.

Discussed with Rong offline, we can reproduce it with a instrumented clang+compiler-rt and run clang -fsanitize=fuzzer test.c.

The compiler cannot resolve the __llvm_profile_instrument_target symbol inside libclang_rt.fuzzer-x86_64.a and we need to manually pass -fprofile-generate to fuzzer.

[xur]

The problem here is libclang_rt.fuzzer-x86_64.a is instrumented and contains calls to profile runtime functions like llvm_profile_instrument_target.
When building with fuzzing, libclang_rt.fuzzer-x86_64.a is linked in. We also have the definition of llvm_profile_instrument_target in the compiler.
But I don't think the compiler knows how to resolve this symbol to itself.

I think this could happen to any library, not just compiler runtime.

It seems to me if one builds an library with instrumentation, one needs to link the profile-runtime library manually.
For this case, if I add -fprofile-generate to the fuzzer, the build would pass.

Do you have to build compiler_rt in instrumentation build? Is it a way to avoid?

If not, this patch certainly would help. Especially this is under an option and sets to false by default.

[zhizhouy]

Do you have to build compiler_rt in instrumentation build? Is it a way to avoid?

Specifically in ChromeOS, due to build system limit for now, we have to build compiler_rt with clang at the same time and only once.

If we only build an instrumented clang, ChromeOS will remove the bootstrap version compiler_rt after the build, thus lead to a broken toolchain.

Because of the behavior of ChromeOS, I would say this patch would help.

[xur]

One correction to my previous comment: the default of NOT COMPILER_RT_ENABLE_PGO is true. But I think it's OK, as far as we have a way to build the instrumented version of compiler_rt.

Please do indicate this (the change of default behavior) in your commit message.

This patch looks good to me.

compiler-rt/CMakeLists.txt

 append_list_if(COMPILER_RT_HAS_FNO_STACK_PROTECTOR_FLAG -fno-stack-protector SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_FNO_SANITIZE_SAFE_STACK_FLAG -fno-sanitize=safe-stack SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_FVISIBILITY_HIDDEN_FLAG -fvisibility=hidden SANITIZER_COMMON_CFLAGS)
 if(NOT COMPILER_RT_HAS_FVISIBILITY_HIDDEN_FLAG)
   append_list_if(COMPILER_RT_HAS_FVISIBILITY_INLINES_HIDDEN_FLAG -fvisibility-inlines-hidden SANITIZER_COMMON_CFLAGS)
 endif()
 append_list_if(COMPILER_RT_HAS_FNO_LTO_FLAG -fno-lto SANITIZER_COMMON_CFLAGS)
 
+# By default do not instrument or use profdata for compiler-rt.
+if(NOT COMPILER_RT_ENABLE_PGO)
+  if(LLVM_PROFDATA_FILE AND COMPILER_RT_HAS_FNO_PROFILE_INSTR_USE_FLAG)
+    list(APPEND SANITIZER_COMMON_CFLAGS "-fno-profile-instr-use")
+  endif()
+  if(LLVM_BUILD_INSTRUMENTED MATCHES IR AND COMPILER_RT_HAS_FNO_PROFILE_GENERATE_FLAG)
+    list(APPEND SANITIZER_COMMON_CFLAGS "-fno-profile-generate")
+  elseif(LLVM_BUILD_INSTRUMENTED AND COMPILER_RT_HAS_FNO_PROFILE_INSTR_GENERATE_FLAG)
+    list(APPEND SANITIZER_COMMON_CFLAGS "-fno-profile-instr-generate")
+  endif()
+endif()
+
 # The following is a workaround for powerpc64le. This is the only architecture
 # that requires -fno-function-sections to work properly. If lacking, the ASan
 # Linux test function-sections-are-bad.cpp fails with the following error:
 # 'undefined symbol: __sanitizer_unaligned_load32'.
 if(DEFINED TARGET_powerpc64le_CFLAGS)
   if(CMAKE_CXX_COMPILER_ID MATCHES "XL")
     append("-qnofuncsect" TARGET_powerpc64le_CFLAGS)
   else()

compiler-rt/cmake/Modules/AddCompilerRT.cmake

function(add_compiler_rt_runtime name type)
   # Until we support this some other way, build compiler-rt runtime without LTO
   # to allow non-LTO projects to link with it.
   if(COMPILER_RT_HAS_FNO_LTO_FLAG)
     set(NO_LTO_FLAGS "-fno-lto")
   else()
     set(NO_LTO_FLAGS "")
   endif()
 
+  # By default do not instrument or use profdata for compiler-rt.
+  set(NO_PGO_FLAGS "")
+  if(NOT COMPILER_RT_ENABLE_PGO)
+    if(LLVM_PROFDATA_FILE AND COMPILER_RT_HAS_FNO_PROFILE_INSTR_USE_FLAG)
+      list(APPEND NO_PGO_FLAGS "-fno-profile-instr-use")
+    endif()
+    if(LLVM_BUILD_INSTRUMENTED MATCHES IR AND COMPILER_RT_HAS_FNO_PROFILE_GENERATE_FLAG)
+      list(APPEND NO_PGO_FLAGS "-fno-profile-generate")
+    elseif(LLVM_BUILD_INSTRUMENTED AND COMPILER_RT_HAS_FNO_PROFILE_INSTR_GENERATE_FLAG)
+      list(APPEND NO_PGO_FLAGS "-fno-profile-instr-generate")
+    endif()
+  endif()
+
   list(LENGTH LIB_SOURCES LIB_SOURCES_LENGTH)
   if (${LIB_SOURCES_LENGTH} GREATER 0)
     # Add headers to LIB_SOURCES for IDEs. It doesn't make sense to
     # do this for a runtime library that only consists of OBJECT
     # libraries, so only add the headers when source files are present.
     compiler_rt_process_sources(LIB_SOURCES
       ${LIB_SOURCES}
       ADDITIONAL_HEADERS
foreach(os ${LIB_OS})
         set(libname "${name}_${os}")
       else()
         set(libname "${name}_${os}_dynamic")
         set(extra_link_flags_${libname} ${DARWIN_${os}_LINK_FLAGS} ${LIB_LINK_FLAGS})
       endif()
       list_intersect(LIB_ARCHS_${libname} DARWIN_${os}_ARCHS LIB_ARCHS)
       if(LIB_ARCHS_${libname})
         list(APPEND libnames ${libname})
-        set(extra_cflags_${libname} ${DARWIN_${os}_CFLAGS} ${NO_LTO_FLAGS} ${LIB_CFLAGS})
+        set(extra_cflags_${libname} ${DARWIN_${os}_CFLAGS} ${NO_LTO_FLAGS} ${NO_PGO_FLAGS} ${LIB_CFLAGS})
         set(output_name_${libname} ${libname}${COMPILER_RT_OS_SUFFIX})
         set(sources_${libname} ${LIB_SOURCES})
         format_object_libs(sources_${libname} ${os} ${LIB_OBJECT_LIBS})
         get_compiler_rt_output_dir(${COMPILER_RT_DEFAULT_TARGET_ARCH} output_dir_${libname})
         get_compiler_rt_install_dir(${COMPILER_RT_DEFAULT_TARGET_ARCH} install_dir_${libname})
       endif()
     endforeach()
   else()
foreach(arch ${LIB_ARCHS})
           set_output_name(output_name_${libname} ${name}_dynamic ${arch})
         else()
           set_output_name(output_name_${libname} ${name} ${arch})
         endif()
       endif()
       set(sources_${libname} ${LIB_SOURCES})
       format_object_libs(sources_${libname} ${arch} ${LIB_OBJECT_LIBS})
       set(libnames ${libnames} ${libname})
-      set(extra_cflags_${libname} ${TARGET_${arch}_CFLAGS} ${NO_LTO_FLAGS} ${LIB_CFLAGS})
+      set(extra_cflags_${libname} ${TARGET_${arch}_CFLAGS} ${NO_LTO_FLAGS} ${NO_PGO_FLAGS} ${LIB_CFLAGS})
       get_compiler_rt_output_dir(${arch} output_dir_${libname})
       get_compiler_rt_install_dir(${arch} install_dir_${libname})
     endforeach()
   endif()
 
   if(NOT libnames)
     return()
   endif()

compiler-rt/cmake/config-ix.cmake

 check_cxx_compiler_flag(-fno-sanitize=safe-stack COMPILER_RT_HAS_FNO_SANITIZE_SAFE_STACK_FLAG)
 check_cxx_compiler_flag(-fvisibility=hidden  COMPILER_RT_HAS_FVISIBILITY_HIDDEN_FLAG)
 check_cxx_compiler_flag(-frtti               COMPILER_RT_HAS_FRTTI_FLAG)
 check_cxx_compiler_flag(-fno-rtti            COMPILER_RT_HAS_FNO_RTTI_FLAG)
 check_cxx_compiler_flag("-Werror -fno-function-sections" COMPILER_RT_HAS_FNO_FUNCTION_SECTIONS_FLAG)
 check_cxx_compiler_flag(-std=c++14           COMPILER_RT_HAS_STD_CXX14_FLAG)
 check_cxx_compiler_flag(-ftls-model=initial-exec COMPILER_RT_HAS_FTLS_MODEL_INITIAL_EXEC)
 check_cxx_compiler_flag(-fno-lto             COMPILER_RT_HAS_FNO_LTO_FLAG)
+check_cxx_compiler_flag(-fno-profile-generate COMPILER_RT_HAS_FNO_PROFILE_GENERATE_FLAG)
+check_cxx_compiler_flag(-fno-profile-instr-generate COMPILER_RT_HAS_FNO_PROFILE_INSTR_GENERATE_FLAG)
+check_cxx_compiler_flag(-fno-profile-instr-use COMPILER_RT_HAS_FNO_PROFILE_INSTR_USE_FLAG)
 check_cxx_compiler_flag("-Werror -msse3" COMPILER_RT_HAS_MSSE3_FLAG)
 check_cxx_compiler_flag("-Werror -msse4.2"   COMPILER_RT_HAS_MSSE4_2_FLAG)
 check_cxx_compiler_flag(--sysroot=.          COMPILER_RT_HAS_SYSROOT_FLAG)
 check_cxx_compiler_flag("-Werror -mcrc"      COMPILER_RT_HAS_MCRC_FLAG)
 check_cxx_compiler_flag(-fno-partial-inlining COMPILER_RT_HAS_FNO_PARTIAL_INLINING_FLAG)
 
 if(NOT WIN32 AND NOT CYGWIN)
   # MinGW warns if -fvisibility-inlines-hidden is used.

compiler-rt/lib/crt/CMakeLists.txt

function(check_cxx_section_exists section output)
   string(REGEX MATCHALL "<[A-Za-z0-9_]*>" substitutions
          ${CMAKE_C_COMPILE_OBJECT})
 
   set(try_compile_flags "${ARG_FLAGS}")
   if(CMAKE_C_COMPILER_ID MATCHES Clang AND CMAKE_C_COMPILER_TARGET)
     list(APPEND try_compile_flags "-target ${CMAKE_C_COMPILER_TARGET}")
   endif()
   append_list_if(COMPILER_RT_HAS_FNO_LTO_FLAG -fno-lto try_compile_flags)
+  if(NOT COMPILER_RT_ENABLE_PGO)
+    if(LLVM_PROFDATA_FILE AND COMPILER_RT_HAS_FNO_PROFILE_INSTR_USE_FLAG)
+      list(APPEND try_compile_flags "-fno-profile-instr-use")
+    endif()
+    if(LLVM_BUILD_INSTRUMENTED MATCHES IR AND COMPILER_RT_HAS_FNO_PROFILE_GENERATE_FLAG)
+      list(APPEND try_compile_flags "-fno-profile-generate")
+    elseif(LLVM_BUILD_INSTRUMENTED AND COMPILER_RT_HAS_FNO_PROFILE_INSTR_GENERATE_FLAG)
+      list(APPEND try_compile_flags "-fno-profile-instr-generate")
+    endif()
+  endif()
 
   string(REPLACE ";" " " extra_flags "${try_compile_flags}")
 
   set(test_compile_command "${CMAKE_C_COMPILE_OBJECT}")
   foreach(substitution ${substitutions})
     if(substitution STREQUAL "<CMAKE_C_COMPILER>")
       string(REPLACE "<CMAKE_C_COMPILER>" "${CMAKE_C_COMPILER} ${CMAKE_C_COMPILER_ARG1}"
              test_compile_command ${test_compile_command})