tsan: strip top inlined internal frames

[dvyukov]

Summary

The new GET_CURRENT_PC() can lead to spurious top inlined internal frames.
Here are 2 examples from bots, in both cases the malloc is supposed to be
the top frame (#0):

WARNING: ThreadSanitizer: signal-unsafe call inside of a signal
  #0 __sanitizer::StackTrace::GetNextInstructionPc(unsigned long)
  #1 malloc

Location is heap block of size 99 at 0xbe3800003800 allocated by thread T1:
  #0 __sanitizer::StackTrace::GetNextInstructionPc(unsigned long)
  #1 malloc

Let's strip these internal top frames from reports.
With other code changes I also observed some top frames
from __tsan::ScopedInterceptor, proactively remove these as well.

[dvyukov]

I've landed this to unbreak bots and to unblock https://reviews.llvm.org/D106078 which unbreaks more bots. Please post-review.

[dyung]

I think additional exclusions are needed, we hit this problem on our internal bot, and I don't think this change will cover all of them. A quick check shows we also probably need __tsan::cur_thread_init.

[dvyukov]

In D106081#2880808, @dyung wrote:

I think additional exclusions are needed, we hit this problem on our internal bot, and I don't think this change will cover all of them. A quick check shows we also probably need __tsan::cur_thread_init.

I've sent D106088. Thanks for reporting.

[dyung]

Even with this change applied, we are still seeing 3 tests fail with __tsan::ScopedInterceptor in the stack. For example:

FAIL: SanitizerCommon-tsan-x86_64-Linux :: Linux/aligned_alloc-alignment.cpp (3254 of 83427)
******************** TEST 'SanitizerCommon-tsan-x86_64-Linux :: Linux/aligned_alloc-alignment.cpp' FAILED ********************
Script:
--
: 'RUN: at line 1';      /home/jenkins/j/w/workspace/opensource/opensource_build/build/./bin/clang  --driver-mode=g++ -gline-tables-only -fsanitize=thread  -m64 -funwind-tables  -ldl  -O0 /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/test/sanitizer_common/TestCases/Linux/aligned_alloc-alignment.cpp -o /home/jenkins/j/w/workspace/opensource/opensource_build/build/projects/compiler-rt/test/sanitizer_common/tsan-x86_64-Linux/Linux/Output/aligned_alloc-alignment.cpp.tmp
: 'RUN: at line 4';   env TSAN_OPTIONS=allocator_may_return_null=0 not  /home/jenkins/j/w/workspace/opensource/opensource_build/build/projects/compiler-rt/test/sanitizer_common/tsan-x86_64-Linux/Linux/Output/aligned_alloc-alignment.cpp.tmp 17 2>&1 | FileCheck /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/test/sanitizer_common/TestCases/Linux/aligned_alloc-alignment.cpp
: 'RUN: at line 6';   env TSAN_OPTIONS=allocator_may_return_null=0 not  /home/jenkins/j/w/workspace/opensource/opensource_build/build/projects/compiler-rt/test/sanitizer_common/tsan-x86_64-Linux/Linux/Output/aligned_alloc-alignment.cpp.tmp 8 2>&1 | FileCheck /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/test/sanitizer_common/TestCases/Linux/aligned_alloc-alignment.cpp
: 'RUN: at line 8';   env TSAN_OPTIONS=allocator_may_return_null=0 not  /home/jenkins/j/w/workspace/opensource/opensource_build/build/projects/compiler-rt/test/sanitizer_common/tsan-x86_64-Linux/Linux/Output/aligned_alloc-alignment.cpp.tmp 0 2>&1 | FileCheck /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/test/sanitizer_common/TestCases/Linux/aligned_alloc-alignment.cpp
: 'RUN: at line 11';   env TSAN_OPTIONS=allocator_may_return_null=1  /home/jenkins/j/w/workspace/opensource/opensource_build/build/projects/compiler-rt/test/sanitizer_common/tsan-x86_64-Linux/Linux/Output/aligned_alloc-alignment.cpp.tmp 17 2>&1 | FileCheck /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/test/sanitizer_common/TestCases/Linux/aligned_alloc-alignment.cpp --check-prefix=CHECK-NULL
: 'RUN: at line 12';   env TSAN_OPTIONS=allocator_may_return_null=1  /home/jenkins/j/w/workspace/opensource/opensource_build/build/projects/compiler-rt/test/sanitizer_common/tsan-x86_64-Linux/Linux/Output/aligned_alloc-alignment.cpp.tmp 8 2>&1 | FileCheck /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/test/sanitizer_common/TestCases/Linux/aligned_alloc-alignment.cpp --check-prefix=CHECK-NULL
: 'RUN: at line 13';   env TSAN_OPTIONS=allocator_may_return_null=1  /home/jenkins/j/w/workspace/opensource/opensource_build/build/projects/compiler-rt/test/sanitizer_common/tsan-x86_64-Linux/Linux/Output/aligned_alloc-alignment.cpp.tmp 0 2>&1 | FileCheck /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/test/sanitizer_common/TestCases/Linux/aligned_alloc-alignment.cpp --check-prefix=CHECK-NULL
--
Exit Code: 1

Command Output (stderr):
--
/home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/test/sanitizer_common/TestCases/Linux/aligned_alloc-alignment.cpp:34:12: error: CHECK: expected string not found in input
 // CHECK: {{#[12] .*main .*aligned_alloc-alignment.cpp:}}[[@LINE-4]]
           ^
<stdin>:2:160: note: scanning from here
 #0 __tsan::ScopedInterceptor::EnableIgnores() /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:280:28 (aligned_alloc-alignment.cpp.tmp+0x45a210)
                                                                                                                                                               ^
<stdin>:2:160: note: with "@LINE-4" equal to "30"
 #0 __tsan::ScopedInterceptor::EnableIgnores() /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:280:28 (aligned_alloc-alignment.cpp.tmp+0x45a210)
                                                                                                                                                               ^
<stdin>:7:82: note: possible intended match here
 #5 main /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/test/sanitizer_common/TestCases/Linux/aligned_alloc-alignment.cpp:30:13 (aligned_alloc-alignment.cpp.tmp+0x4d4b62)
                                                                                 ^

Input file: <stdin>
Check file: /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/test/sanitizer_common/TestCases/Linux/aligned_alloc-alignment.cpp

-dump-input=help explains the following input dump.

Input was:
<<<<<<
            1: ==1389839==ERROR: ThreadSanitizer: invalid alignment requested in aligned_alloc: 17, alignment must be a power of two and the requested size 0x64 must be a multiple of alignment 
            2:  #0 __tsan::ScopedInterceptor::EnableIgnores() /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:280:28 (aligned_alloc-alignment.cpp.tmp+0x45a210) 
check:34'0                                                                                                                                                                    X~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
check:34'1                                                                                                                                                                                                  with "@LINE-4" equal to "30"
            3:  #1 __tsan::ScopedInterceptor::EnableIgnores() /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:274:6 (aligned_alloc-alignment.cpp.tmp+0x45a210) 
check:34'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            4:  #2 __tsan::ScopedInterceptor::ScopedInterceptor(__tsan::ThreadState*, char const*, unsigned long) /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:261:16 (aligned_alloc-alignment.cpp.tmp+0x45a210) 
check:34'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            5:  #3 __tsan::ScopedInterceptor::ScopedInterceptor(__tsan::ThreadState*, char const*, unsigned long) /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:251:1 (aligned_alloc-alignment.cpp.tmp+0x45a210) 
check:34'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            6:  #4 __interceptor_aligned_alloc /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:810:3 (aligned_alloc-alignment.cpp.tmp+0x45a210) 
check:34'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            7:  #5 main /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/test/sanitizer_common/TestCases/Linux/aligned_alloc-alignment.cpp:30:13 (aligned_alloc-alignment.cpp.tmp+0x4d4b62) 
check:34'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
check:34'2                                                                                      ?                                                                                             possible intended match
            8:  #6 __libc_start_main /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16 (libc.so.6+0x270b2) 
check:34'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            9:  
check:34'0     ~
           10: ==1389839==HINT: if you don't care about these errors you may set allocator_may_return_null=1 
check:34'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
           11: SUMMARY: ThreadSanitizer: invalid-aligned-alloc-alignment /home/jenkins/j/w/p/cpu-toolchain-ppr/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:280:28 in __tsan::ScopedInterceptor::EnableIgnores() 
check:34'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>

Does the fix need to be applied somewhere else as well perhaps?

compiler-rt/lib/tsan/rtl/tsan_rtl_report.cpp

 }
 #endif
 
 SANITIZER_WEAK_DEFAULT_IMPL
 void __tsan_on_report(const ReportDesc *rep) {
   (void)rep;
 }
 
-static void StackStripMain(SymbolizedStack *frames) {
+bool InternalFrame(const char *func) {
+  static const char *frames[] = {
+      "__tsan::ScopedInterceptor",
+      "__sanitizer::StackTrace",
+  };
+  for (auto frame : frames) {
+    if (!internal_strncmp(func, frame, internal_strlen(frame)))
+      return true;
+  }
+  return false;
+}
+
+static SymbolizedStack *StackStripMain(SymbolizedStack *frames) {
+  for (; frames && frames->info.function; frames = frames->next) {
+    // Remove top inlined frames from our interceptors.
+    if (!InternalFrame(frames->info.function))
+      break;
+  }
   SymbolizedStack *last_frame = nullptr;
   SymbolizedStack *last_frame2 = nullptr;
   for (SymbolizedStack *cur = frames; cur; cur = cur->next) {
     last_frame2 = last_frame;
     last_frame = cur;
   }
 
   if (last_frame2 == 0)
-    return;
+    return frames;
 #if !SANITIZER_GO
   const char *last = last_frame->info.function;
   const char *last2 = last_frame2->info.function;
   // Strip frame above 'main'
   if (last2 && 0 == internal_strcmp(last2, "main")) {
     last_frame->ClearAll();
     last_frame2->next = nullptr;
   // Strip our internal thread start routine.
   } else if (last && 0 == internal_strcmp(last, "__tsan_thread_start_func")) {
     last_frame->ClearAll();
     last_frame2->next = nullptr;
   // Strip global ctors init.
-  } else if (last && 0 == internal_strcmp(last, "__do_global_ctors_aux")) {
+  } else if (last && (0 == internal_strcmp(last, "__do_global_ctors_aux") ||
+                      0 == internal_strcmp(last, "__libc_csu_init"))) {
     last_frame->ClearAll();
     last_frame2->next = nullptr;
   // If both are 0, then we probably just failed to symbolize.
   } else if (last || last2) {
     // Ensure that we recovered stack completely. Trimmed stack
     // can actually happen if we do not instrument some code,
     // so it's only a debug print. However we must try hard to not miss it
     // due to our fault.
     DPrintf("Bottom stack frame is missed\n");
   }
 #else
   // The last frame always point into runtime (gosched0, goexit0, runtime.main).
   last_frame->ClearAll();
   last_frame2->next = nullptr;
 #endif
+  return frames;
 }
 
 ReportStack *SymbolizeStackId(u32 stack_id) {
   if (stack_id == 0)
     return 0;
   StackTrace stack = StackDepotGet(stack_id);
   if (stack.trace == nullptr)
     return nullptr;
for (uptr si = 0; si < trace.size; si++) {
     while (last->next) {
       last->info.address = pc;  // restore original pc for report
       last = last->next;
     }
     last->info.address = pc;  // restore original pc for report
     last->next = top;
     top = ent;
   }
-  StackStripMain(top);
-
   ReportStack *stack = ReportStack::New();
-  stack->frames = top;
+  stack->frames = StackStripMain(top);
   return stack;
 }
 
 bool ShouldReport(ThreadState *thr, ReportType typ) {
   // We set thr->suppress_reports in the fork context.
   // Taking any locking in the fork context can lead to deadlocks.
   // If any locks are already taken, it's too late to do this check.
   CheckNoLocks(thr);