[asan] Add CMake hook to override shadow scale in compiler_rt

[waltl]

Summary

Allow user to override shadow scale in compiler_rt by passing
-DCOMPILER_RT_ASAN_SHADOW_SCALE=n to CMake. Propagate the override
shadow scale value via a compiler define to compiler-rt and asan
tests. Tests will use the define to partially disable unsupported
tests. Set "-mllvm -asan-mapping-scale=<n>" for compiler_rt tests.

[waltl]

Add "differential revision" line.

[waltl]

Revert mistaken update.

[waltl]

Miscellaneous cleanups.

[vitalybuka]

Can you explain or point to use-case where 5 is better than 3?

[waltl]

In D39469#919730, @vitalybuka wrote:

Can you explain or point to use-case where 5 is better than 3?

See http://lists.llvm.org/pipermail/llvm-dev/2017-October/118679.html for background. The end-goal for me is to port ASan to an embedding platform that has no virtual memory and limited memory, where reserving 12.5% of the memory for shadow memory is too high. Getting this to work for x86 is only a small amount of extra work and gives me a way to test/validate the changes in the community.

[vitalybuka]

I think LLVM_ASAN_SHADOW_SCALE is not best approach.
Seems unusual that compiler which can generate code for multiple platforms needs to be reconfigure and rebuild.

@kcc already suggested using regular compile flag: http://lists.llvm.org/pipermail/llvm-dev/2017-October/118685.html
So clang will be able to generate both versions of instrumented code. Obviously we need to have multiple version of run-time so driver will pick proper version based on -fsanitize-address-granularity value. (or somehow we can init the same runtime correctly, not sure yet if it's possible)

[vitalybuka]

Just talked with @eugenis
Maybe simpler approach is following:

1. we have llvm flag in AddressSanitizer.cpp which deppends on platform, 8 or 32
2. runtime also will pick 8 or 32 from platfor
3. if you'd like to setup build bot on e.g. x86 just to test 32, than you call add override flag like this. but it should be in compiler-rt only: e.g. COMPILER_RT_ASAN_SHADOW_GRANULARITY

This flag can switch granularity of runtime and also add -mllvm -asan-shadow-granularity=32 for tests

[waltl]

In D39469#919760, @vitalybuka wrote:

Just talked with @eugenis
Maybe simpler approach is following:

1. we have llvm flag in AddressSanitizer.cpp which deppends on platform, 8 or 32
2. runtime also will pick 8 or 32 from platfor
3. if you'd like to setup build bot on e.g. x86 just to test 32, than you call add override flag like this. but it should be in compiler-rt only: e.g. COMPILER_RT_ASAN_SHADOW_GRANULARITY

This flag can switch granularity of runtime and also add -mllvm -asan-shadow-granularity=32 for tests

That sounds good; I will work on it. One followup question: the current flag is called -asan-shadow-scale. Is that ok or should I change that to -asan-shadow-granularity? I don't know if there are any users who depend on the existing name.

[waltl]

In D39469#919858, @waltl wrote:

That sounds good; I will work on it. One followup question: the current flag is called -asan-shadow-scale. Is that ok or should I change that to -asan-shadow-granularity? I don't know if there are any users who depend on the existing name.

Correction: current flag is called -asan-mapping-scale; there is also an -asan-mapping-offset flag for changing the shadow offset.

[vitalybuka]

I'd keep it as is --asan-mapping-scale. I noticed that you are using this in other changes, but didn't realize that it's already there.

[kcc]

--asan-maping-scale is an -mmlv flag.
Such flags should not be given to users -- they are internal testing only flags.

All this #ifdef stuff really sucks. I don't claim I immediately see a better way to do this, but pleeeeeease, try to avoid such ifdefs when possible.

[kcc]

--asan-maping-scale is an -mmlv flag.

-mllvm flag

[vitalybuka]

In D39469#919883, @kcc wrote:

--asan-maping-scale is an -mmlv flag.

-mllvm flag

Users do not need this flag. Platform will control granularity. So this flag is going to be used by bots (llvm and compiler-rt tests), when we enforce granularity unusual for host platform.

[waltl]

Change approach to only override shadow scale in compiler_rt

[vitalybuka]

compiler-rt/CMakeLists.txt

51 ↗

(On Diff #122458)

if (COMPILER_RT_ASAN_SHADOW_SCALE GREATER 8 OR
         COMPILER_RT_ASAN_SHADOW_SCALE LESS 0)

58 ↗

(On Diff #122458)

I'd expect following should work:
-mllvm -asan-mapping-scale=${COMPILER_RT_ASAN_SHADOW_SCALE}

compiler-rt/lib/asan/asan_mapping.h

135 ↗

(On Diff #122458)

I assume that default for your platform will be changed in separate CL?

compiler-rt/test/lit.common.configured.in

28 ↗

(On Diff #122458)

can you just just use target_cflags and COMPILER_RT_TEST_COMPILER_CFLAGS

[waltl]

compiler-rt/CMakeLists.txt

51 ↗

(On Diff #122458)

Note, however, that this will also accept non-numbers like "foo".

58 ↗

(On Diff #122458)

You are right. I'll make the change.

compiler-rt/lib/asan/asan_mapping.h

135 ↗

(On Diff #122458)

Yes -- down the road when I upstream my port, which doesn't exist yet.

compiler-rt/test/lit.common.configured.in

28 ↗

(On Diff #122458)

Maybe? Currently COMPILER_RT_TEST_COMPILER_CFLAGS doesn't get propagated to target_cflags because individual sanitizers have their own definitions of target_cflags that supercede it. Is this intended? If the right thing to do is to append the target_cflags together I can do that and it'd better serve our purpose here.

[vitalybuka]

compiler-rt/test/lit.common.configured.in

28 ↗

(On Diff #122458)

They append content of COMPILER_RT_TEST_COMPILER_CFLAGS, it's in get_test_cc_for_arch implementation

[waltl]

compiler-rt/test/lit.common.configured.in

28 ↗

(On Diff #122458)

Sorry if I'm missing something, but it looks to me that in get_test_cc_for_arch, COMPILER_RT_TEST_COMPILER_CFLAGS is only used for ANDROID, arm, and aarch64. For others, the target CFLAGS comes from TARGET_${arch}_CFLAGS which is constructed by test_targets(), but I am not seeing COMPILER_RT_TEST_COMPILER_CFLAGS referenced in there. This is also supported by experimentation.

[waltl]

Address CR comments.

[waltl]

I partially addressed this issue by setting config.target_flags in compiler-rt/test/lit.common.cfg. This still requires config.asan_shadow_scale to be set but we need that to set a feature anyways (https://reviews.llvm.org/D39772).

compiler-rt/trunk/CMakeLists.txt

 mark_as_advanced(COMPILER_RT_BUILD_XRAY)
 option(COMPILER_RT_BUILD_LIBFUZZER "Build libFuzzer" ON)
 mark_as_advanced(COMPILER_RT_BUILD_LIBFUZZER)
 option(COMPILER_RT_BUILD_PROFILE "Build profile runtime" ON)
 mark_as_advanced(COMPILER_RT_BUILD_PROFILE)
 option(COMPILER_RT_BUILD_XRAY_NO_PREINIT "Build xray with no preinit patching" OFF)
 mark_as_advanced(COMPILER_RT_BUILD_XRAY_NO_PREINIT)
 
+set(COMPILER_RT_ASAN_SHADOW_SCALE ""
+    CACHE STRING "Override the shadow scale to be used in ASan runtime")
+
+if (NOT COMPILER_RT_ASAN_SHADOW_SCALE STREQUAL "")
+  # Check that the shadow scale value is valid.
+  if (NOT (COMPILER_RT_ASAN_SHADOW_SCALE GREATER -1 AND
+           COMPILER_RT_ASAN_SHADOW_SCALE LESS 8))
+    message(FATAL_ERROR "
+      Invalid ASan Shadow Scale '${COMPILER_RT_ASAN_SHADOW_SCALE}'.")
+  endif()
+
+  set(COMPILER_RT_ASAN_SHADOW_SCALE_LLVM_FLAG
+      -mllvm -asan-mapping-scale=${COMPILER_RT_ASAN_SHADOW_SCALE})
+  set(COMPILER_RT_ASAN_SHADOW_SCALE_DEFINITION
+      ASAN_SHADOW_SCALE=${COMPILER_RT_ASAN_SHADOW_SCALE})
+  set(COMPILER_RT_ASAN_SHADOW_SCALE_FLAG
+      -D${COMPILER_RT_ASAN_SHADOW_SCALE_DEFINITION})
+endif()
+
 set(COMPILER_RT_BAREMETAL_BUILD OFF CACHE BOOLEAN
   "Build for a bare-metal target.")
 
 if (COMPILER_RT_STANDALONE_BUILD)
   load_llvm_config()
 
   # Find Python interpreter.
   set(Python_ADDITIONAL_VERSIONS 2.7 2.6 2.5)

compiler-rt/trunk/lib/asan/CMakeLists.txt

set(ASAN_CXX_SOURCES
   asan_new_delete.cc)
 
 set(ASAN_PREINIT_SOURCES
   asan_preinit.cc)
 
 include_directories(..)
 
 set(ASAN_CFLAGS ${SANITIZER_COMMON_CFLAGS})
+set(ASAN_COMMON_DEFINITIONS ${COMPILER_RT_ASAN_SHADOW_SCALE_DEFINITION})
 
 append_rtti_flag(OFF ASAN_CFLAGS)
 
 set(ASAN_DYNAMIC_LINK_FLAGS ${SANITIZER_COMMON_LINK_FLAGS})
 
 if(ANDROID)
 # On Android, -z global does not do what it is documented to do.
 # On Android, -z global moves the library ahead in the lookup order,

compiler-rt/trunk/lib/asan/asan_mapping.h

 // (the exact location of HighShadow/HighMem may vary depending
 //  on WoW64, /LARGEADDRESSAWARE, etc).
 // || `[0x50000000, 0xffffffff]` || HighMem    ||
 // || `[0x3a000000, 0x4fffffff]` || HighShadow ||
 // || `[0x36000000, 0x39ffffff]` || ShadowGap  ||
 // || `[0x30000000, 0x35ffffff]` || LowShadow  ||
 // || `[0x00000000, 0x2fffffff]` || LowMem     ||
 
+#if defined(ASAN_SHADOW_SCALE)
+static const u64 kDefaultShadowScale = ASAN_SHADOW_SCALE;
+#else
 static const u64 kDefaultShadowScale = 3;
+#endif
 static const u64 kDefaultShadowSentinel = ~(uptr)0;
 static const u64 kDefaultShadowOffset32 = 1ULL << 29;  // 0x20000000
 static const u64 kDefaultShadowOffset64 = 1ULL << 44;
 static const u64 kDefaultShort64bitShadowOffset = 0x7FFF8000;  // < 2G.
 static const u64 kIosShadowOffset32 = 1ULL << 30;  // 0x40000000
 static const u64 kIosShadowOffset64 = 0x120200000;
 static const u64 kIosSimShadowOffset32 = 1ULL << 30;
 static const u64 kIosSimShadowOffset64 = kDefaultShadowOffset64;

compiler-rt/trunk/lib/asan/tests/CMakeLists.txt

 set(ASAN_UNITTEST_HEADERS
   asan_mac_test.h
   asan_test_config.h
   asan_test_utils.h)
 
 set(ASAN_UNITTEST_COMMON_CFLAGS
   ${COMPILER_RT_UNITTEST_CFLAGS}
   ${COMPILER_RT_GTEST_CFLAGS}
+  ${COMPILER_RT_ASAN_SHADOW_SCALE_LLVM_FLAG}
   -I${COMPILER_RT_SOURCE_DIR}/include
   -I${COMPILER_RT_SOURCE_DIR}/lib
   -I${COMPILER_RT_SOURCE_DIR}/lib/asan
   -I${COMPILER_RT_SOURCE_DIR}/lib/sanitizer_common/tests
   -fno-rtti
   -O2
   -Wno-format
   -Werror=sign-compare

 endif()
 if(MSVC)
   list(APPEND ASAN_UNITTEST_COMMON_CFLAGS -gcodeview)
 endif()
 list(APPEND ASAN_UNITTEST_COMMON_LINK_FLAGS -g)
 
 # Use -D instead of definitions to please custom compile command.
 list(APPEND ASAN_UNITTEST_COMMON_CFLAGS
+  ${COMPILER_RT_ASAN_SHADOW_SCALE_FLAG}
   -DASAN_HAS_BLACKLIST=1
   -DASAN_HAS_EXCEPTIONS=1
   -DASAN_UAR=0)
 
 if(APPLE)
   list(APPEND ASAN_UNITTEST_COMMON_CFLAGS ${DARWIN_osx_CFLAGS})
   list(APPEND ASAN_UNITTEST_COMMON_LINK_FLAGS ${DARWIN_osx_LINK_FLAGS})
 

compiler-rt/trunk/test/lit.common.cfg

 elif compiler_id == 'GNU':
   config.cxx_mode_flags = ["-x c++"]
   config.debug_info_flags = ["-g"]
 else:
   lit_config.fatal("Unsupported compiler id: %r" % compiler_id)
 # Add compiler ID to the list of available features.
 config.available_features.add(compiler_id)
 
+# If needed, add cflag for shadow scale.
+if config.asan_shadow_scale != '':
+  config.target_cflags += " -mllvm -asan-mapping-scale=" + config.asan_shadow_scale
+
 # BFD linker in 64-bit android toolchains fails to find libm.so, which is a
 # transitive shared library dependency (via asan runtime).
 if config.android:
   config.target_cflags += " -pie -fuse-ld=gold -Wl,--enable-new-dtags"
 
 # Clear some environment variables that might affect Clang.
 possibly_dangerous_env_vars = ['ASAN_OPTIONS', 'DFSAN_OPTIONS', 'LSAN_OPTIONS',
                                'MSAN_OPTIONS', 'UBSAN_OPTIONS',

compiler-rt/trunk/test/lit.common.configured.in

 set_default("llvm_shlib_dir", "@LLVM_LIBRARY_OUTPUT_INTDIR@")
 set_default("gold_executable", "@GOLD_EXECUTABLE@")
 set_default("clang", "@COMPILER_RT_RESOLVED_TEST_COMPILER@")
 set_default("compiler_id", "@COMPILER_RT_TEST_COMPILER_ID@")
 set_default("python_executable", "@PYTHON_EXECUTABLE@")
 set_default("compiler_rt_debug", @COMPILER_RT_DEBUG_PYBOOL@)
 set_default("compiler_rt_libdir", "@COMPILER_RT_RESOLVED_LIBRARY_OUTPUT_DIR@")
 set_default("emulator", "@COMPILER_RT_EMULATOR@")
+set_default("asan_shadow_scale", "@COMPILER_RT_ASAN_SHADOW_SCALE@")
 set_default("ios", False)
 set_default("iossim", False)
 set_default("sanitizer_can_use_cxxabi", @SANITIZER_CAN_USE_CXXABI_PYBOOL@)
 set_default("has_lld", @COMPILER_RT_HAS_LLD_PYBOOL@)
 set_default("can_symbolize", @CAN_SYMBOLIZE@)
 set_default("use_lld", False)
 set_default("use_thinlto", False)
 set_default("android", @ANDROID_PYBOOL@)