[sanitizer] Don't use -fomit-frame-pointer in COMPILER_RT_DEBUG=On builds

[kubamracek]

Summary

Using -fomit-frame-pointer sometimes makes a crash log miss some frames. Let's not use this optimization in debug builds.

[filcab]

Hi Kuba,

Do you see this being a problem? On what platform(s)?

Thank you,

Filipe

[kubamracek]

In D20425#434487, @filcab wrote:

Do you see this being a problem? On what platform(s)?

On OS X. Not always, but some stack traces in crash logs (taken by the system's Crash Reporter) are sometimes either truncated or they miss several important frames.

[filcab]

You mean in production, right?
Please add a test that shows the problem. If this problem happens we want
to protect against it being reintroduced in the future.

Thank you,

Filipe

[kubamracek]

In D20425#436476, @filcab wrote:

You mean in production, right?
Please add a test that shows the problem. If this problem happens we want
to protect against it being reintroduced in the future.

Thank you,

Filipe

No, I'm not talking about production. COMPILER_RT_DEBUG=On is not used in production.

Adding a testcase for this is very problematic. I'm talking about external (non-LLVM) tools having issues taking correct backtraces when -fomit-frame-pointers is used. This problem exists both for -O0 (development) and -O3 (production) builds of compiler-rt. This was already discussed a long time ago and the status quo is that -fomit-frame-pointers needs to be used for performance reasons, even when it makes debugging harder and it can break external tools.

This patch changes debug/development builds only, where performance doesn't matter that much, and any optimizations are usually undesirable.

[filcab]

Fair enough, at most we'd know the text works on a given version of Xcode
but wouldn't be able to guarantee to make it work all the time.

I've never seen this problem when trying on OS X, though. What I've seen
are problems around wrap_free and too much data for a given section on tsan.

Thank you,

Filipe

[kubamracek]

In D20425#436478, @filcab wrote:

I've never seen this problem when trying on OS X, though. What I've seen
are problems around wrap_free and too much data for a given section on tsan.

Can you describe those issues? Are they only on OS X?

[filcab]

I've seen them on OS X. Example:

FAIL: AddressSanitizer-x86_64-darwin ::
TestCases/malloc_context_size.cc (1127 of 1216)

• TEST 'AddressSanitizer-x86_64-darwin ::

TestCases/malloc_context_size.cc' FAILED ****

Script:

/Users/filcab/work/llvm-cmake/./bin/clang --driver-mode=g++
-fsanitize=address -mno-omit-leaf-frame-pointer
-fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only
-arch x86_64 -O0
/Users/filcab/work/llvm/projects/compiler-rt/test/asan/TestCases/malloc_context_size.cc
-o /Users/filcab/work/llvm-cmake/projects/compiler-rt/test/asan/X86_64DarwinConfig/TestCases/Output/malloc_context_size.cc.tmp
env ASAN_OPTIONS=abort_on_error=0:log_to_syslog=0:malloc_context_size=0:fast_unwind_on_malloc=0
not /Users/filcab/work/llvm-cmake/projects/compiler-rt/test/asan/X86_64DarwinConfig/TestCases/Output/malloc_context_size.cc.tmp
2>&1 | FileCheck
/Users/filcab/work/llvm/projects/compiler-rt/test/asan/TestCases/malloc_context_size.cc
env ASAN_OPTIONS=abort_on_error=0:log_to_syslog=0:malloc_context_size=0:fast_unwind_on_malloc=1
not /Users/filcab/work/llvm-cmake/projects/compiler-rt/test/asan/X86_64DarwinConfig/TestCases/Output/malloc_context_size.cc.tmp
2>&1 | FileCheck
/Users/filcab/work/llvm/projects/compiler-rt/test/asan/TestCases/malloc_context_size.cc
env ASAN_OPTIONS=abort_on_error=0:log_to_syslog=0:malloc_context_size=1:fast_unwind_on_malloc=0
not /Users/filcab/work/llvm-cmake/projects/compiler-rt/test/asan/X86_64DarwinConfig/TestCases/Output/malloc_context_size.cc.tmp
2>&1 | FileCheck
/Users/filcab/work/llvm/projects/compiler-rt/test/asan/TestCases/malloc_context_size.cc
env ASAN_OPTIONS=abort_on_error=0:log_to_syslog=0:malloc_context_size=1:fast_unwind_on_malloc=1
not /Users/filcab/work/llvm-cmake/projects/compiler-rt/test/asan/X86_64DarwinConfig/TestCases/Output/malloc_context_size.cc.tmp
2>&1 | FileCheck
/Users/filcab/work/llvm/projects/compiler-rt/test/asan/TestCases/malloc_context_size.cc
env ASAN_OPTIONS=abort_on_error=0:log_to_syslog=0:malloc_context_size=2
not /Users/filcab/work/llvm-cmake/projects/compiler-rt/test/asan/X86_64DarwinConfig/TestCases/Output/malloc_context_size.cc.tmp
2>&1 | FileCheck
/Users/filcab/work/llvm/projects/compiler-rt/test/asan/TestCases/malloc_context_size.cc

--check-prefix=TWO

Exit Code: 1

Command Output (stderr):

/Users/filcab/work/llvm/projects/compiler-rt/test/asan/TestCases/malloc_context_size.cc:14:17:
error: expected string not found in input
// CHECK-NEXT: #0 0x{{.*}} in {{operator delete( )?\[\]|wrap__ZdaPv}}

^

<stdin>:10:2: note: scanning from here
#0 0x102993eaf in wrap_free
(/Users/filcab/work/llvm-cmake/lib/clang/3.9.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib+0x4beaf)
^
<stdin>:24:30: note: possible intended match here
0x1c0600001c00: 00 fa fa fa 00 00 00 fa fa fa 00 00 00 fa fa fa

^

---

Testing Time: 596.75s

---

Failing Tests (4):

AddressSanitizer-i386-darwin :: TestCases/large_func_test.cc
AddressSanitizer-i386-darwin :: TestCases/malloc_context_size.cc
AddressSanitizer-x86_64-darwin :: TestCases/large_func_test.cc
AddressSanitizer-x86_64-darwin :: TestCases/malloc_context_size.cc

large_func_test.cc has a similar failure, but it's trying to match
.*_Zna.* and gets wrap_malloc.

Thank you,

Filipe

[kubamracek]

In D20425#437770, @filcab wrote:

...
large_func_test.cc has a similar failure, but it's trying to match
.*_Zna.* and gets wrap_malloc.

Okay, this is completely unrelated to this patch. LLVM-Symbolizer doesn't have issues with missing frame pointers.

What is your OS X version? What is your CMake configure line? What is the output of:

• otool -L /Users/filcab/work/llvm-cmake/lib/clang/3.9.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
• otool -l /Users/filcab/work/llvm-cmake/lib/clang/3.9.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
• otool -L /Users/filcab/work/llvm-cmake/projects/compiler-rt/test/asan/X86_64DarwinConfig/TestCases/Output/malloc_context_size.cc.tmp
• otool -l /Users/filcab/work/llvm-cmake/projects/compiler-rt/test/asan/X86_64DarwinConfig/TestCases/Output/malloc_context_size.cc.tmp

?

[filcab]

What is your OS X version?

[llvm-cmake]% sw_vers
ProductName: Mac OS X
ProductVersion: 10.11.5
BuildVersion: 15F34

What is your CMake configure line?

That has been lost in time. But I usually just add all the dirs for
clang, libcxx, etc. C*FLAGS aren't special, I just set the build mode
to Debug or Release.

What is the output of:

• otool -L /Users/filcab/work/llvm-cmake/lib/clang/3.9.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
• otool -l /Users/filcab/work/llvm-cmake/lib/clang/3.9.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
• otool -L /Users/filcab/work/llvm-cmake/projects/compiler-rt/test/asan/X86_64DarwinConfig/TestCases/Output/malloc_context_size.cc.tmp
• otool -l /Users/filcab/work/llvm-cmake/projects/compiler-rt/test/asan/X86_64DarwinConfig/TestCases/Output/malloc_context_size.cc.tmp

Attaching a text file.

Thank you,

Filipe

BTW, there's also problems with at least one TSan test, in debug
builds (but this one I think I only see it at home (same rough
config)):

• TEST 'ThreadSanitizer-x86_64h :: mutexset7.cc'

FAILED ****

Script:

/Users/filcab/dev/llvm/build-debug/./bin/clang --driver-mode=g++
-fsanitize=thread -Wall -arch x86_64h -gline-tables-only -O1
/Users/filcab/dev/llvm/compiler-rt/test/tsan/mutexset7.cc -o
/Users/filcab/dev/llvm/build-debug/projects/compiler-rt/test/tsan/X86_64HConfig/Output/mutexset7.cc.tmp
&& /Users/filcab/dev/llvm/llvm/projects/compiler-rt/test/tsan/deflake.bash
/Users/filcab/dev/llvm/build-debug/projects/compiler-rt/test/tsan/X86_64HConfig/Output/mutexset7.cc.tmp

| FileCheck /Users/filcab/dev/llvm/compiler-rt/test/tsan/mutexset7.cc

Exit Code: 1

Command Output (stderr):

ld: section DATA/thread_bss extends beyond end of file, file
'/var/folders/9c/8tdsmzsd2cn_c6wkqjbbnd_m0000gn/T/mutexset7-6dfa74.o'
for architecture x86_64h
clang-3.8: error: linker command failed with exit code 1 (use -v to
see invocation)

• bug-in-asan-tests.txt13 KBDownload

[kubamracek]

In D20425#437770, @filcab wrote:

ld: section DATA/thread_bss extends beyond end of file, file
'/var/folders/9c/8tdsmzsd2cn_c6wkqjbbnd_m0000gn/T/mutexset7-6dfa74.o'

This is a bug in the linker in Xcode 7.3. Please upgrade to 7.3.1.

CMakeLists.txt

 if(NOT COMPILER_RT_HAS_FUNC_SYMBOL)
   add_definitions(-D__func__=__FUNCTION__)
 endif()
 
 # Provide some common commmandline flags for Sanitizer runtimes.
 append_list_if(COMPILER_RT_HAS_FPIC_FLAG -fPIC SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_FNO_BUILTIN_FLAG -fno-builtin SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_FNO_EXCEPTIONS_FLAG -fno-exceptions SANITIZER_COMMON_CFLAGS)
+if(NOT COMPILER_RT_DEBUG)
   append_list_if(COMPILER_RT_HAS_FOMIT_FRAME_POINTER_FLAG -fomit-frame-pointer SANITIZER_COMMON_CFLAGS)
+endif()
 append_list_if(COMPILER_RT_HAS_FUNWIND_TABLES_FLAG -funwind-tables SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_FNO_STACK_PROTECTOR_FLAG -fno-stack-protector SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_FNO_SANITIZE_SAFE_STACK_FLAG -fno-sanitize=safe-stack SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_FVISIBILITY_HIDDEN_FLAG -fvisibility=hidden SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_FVISIBILITY_INLINES_HIDDEN_FLAG -fvisibility-inlines-hidden SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_FNO_FUNCTION_SECTIONS_FLAG -fno-function-sections SANITIZER_COMMON_CFLAGS)
 append_list_if(COMPILER_RT_HAS_FNO_LTO_FLAG -fno-lto SANITIZER_COMMON_CFLAGS)