[hwasan] Add __hwasan_add_frame_record to the hwasan interface

[leonardchan]

Summary

Hwasan includes instructions in the prologue that mix the PC and SP and store it into the stack ring buffer stored at __hwasan_tls. This is a thread_local global exposed from the hwasan runtime. However, if TLS-mechanisms or the hwasan runtime haven't been setup yet, it will be invalid to access __hwasan_tls. This is the case for Fuchsia where we instrument libc, so some functions that are instrumented but can run before hwasan initialization will incorrectly access this global. Additionally, libc cannot have any TLS variables, so we cannot weakly define __hwasan_tls until the runtime is loaded.

A way we can work around this is by moving the instructions into a hwasan function that does the store into the ring buffer and creating a weak definition of that function locally in libc. This way __hwasan_tls will not actually be referenced. This is not our long-term solution, but this will allow us to roll out hwasan in the meantime.

This patch includes:

• A new llvm flag for choosing to emit a libcall rather than instructions in the prologue (off by default)
• The libcall for storing into the ringbuffer (__hwasan_record_frame_record)

[vitalybuka]

Could you please move refactoring (like extraction emitPrologue) into a separate CL?

We also need one test for runtime and one test for generated IR.

[leonardchan]

Added tests and this patch doesn't contain refactoring.

[leonardchan]

In D128387#3603205, @vitalybuka wrote:

Could you please move refactoring (like extraction emitPrologue) into a separate CL?

We also need one test for runtime and one test for generated IR.

Should I break this up into 2 compiler-rt and llvm patches?

[mcgrathr]

I don't see a problem having the compiler-rt and compiler changes in the same commit. Seems desirable in this case, actually.

compiler-rt/lib/hwasan/hwasan_interface_internal.h

172

A comment here about when the compiler generates calls to this would be helpful.

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

1210

It seems better to call the subroutine you just broke out above rather than to have its logic still open-coded here as well.
Conversely, perhaps just move this code up before the conditional and reuse the same FrameRecordInfo local variable in both the libcall and direct-store cases, rather than having a subroutine.

[vitalybuka]

In D128387#3623787, @leonardchan wrote:

In D128387#3603205, @vitalybuka wrote:

Could you please move refactoring (like extraction emitPrologue) into a separate CL?

We also need one test for runtime and one test for generated IR.

Should I break this up into 2 compiler-rt and llvm patches?

I don't have preference regarding compiler-rt / llvm split. Seems having them together is useful.
It's more about moving as much as possible into NFC patches. Also in this snapshot it's not obvious what can be move out of the patch.

compiler-rt/lib/hwasan/hwasan.cpp

579

can have a nicer name?
maybe __hwasan_add_frame_record

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

183–187

maybe better to convert hwasan-record-stack-history into enum?
hwasan-record-stack-history-with-calls is meaningless if hwasan-record-stack-history=0

1210

Also looks like can be inserted twice
Value *ThreadLong = IRB.CreateLoad(IntptrTy, SlotPtr);

1228

When I asked about refactoring, you can do all NFC changes in a separate patch,
maybe you can reorder ThreadLongMaybeUntagged calculations in NFC patch so you don't have to touch it here

[leonardchan]

Addressed some comments. Also updated the current stack history codegen path such that getFrameRecordInfo can be reused. Although this changes some test cases. If the patch is too big with this, I think I can move that into a separate one.

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

183–187

Done and updated tests. I couldn't think of a better enum value for the default behavior other than instr :/

1210

It seems better to call the subroutine you just broke out above rather than to have its logic still open-coded here as well. Conversely, perhaps just move this code up before the conditional and reuse the same FrameRecordInfo local variable in both the libcall and direct-store cases, rather than having a subroutine.

Can do. I initially wanted to leave that out since it would mean this patch changes the IR in the prologue slightly and updating some tests that I could save for another patch. Currently, the order is (1) inttoptr (2) or PC, SP (3) store. But reusing the subroutine or FrameRecordInfo would change the order to (1) or SP, PC (2) inttoptr (3) store, but I think that shouldn't affect lowering from IR much.

Also looks like can be inserted twice
Value *ThreadLong = IRB.CreateLoad(IntptrTy, SlotPtr);

I think the second Value *ThreadLong = IRB.CreateLoad(IntptrTy, SlotPtr); in the conditional below shouldn't happen if ThreadLongMaybeUntagged is set, which will be set in this block where the first CreateLoad would happen since it looks like ThreadLongMaybeUntagged = TargetTriple.isAArch64() ? ThreadLong : untagPointer(IRB, ThreadLong) should never be nullptr, unless I'm perhaps missing something.

1228

So this part's a bit awkward bc in the above block, the intermediate variables SlotPtr and ThreadLong are used for a bunch of other things like WrapMask and StackBaseTag which aren't needed here. That is, if ThreadLongMaybeUntagged isn't set above, then neither would SlotPtr or ThreadLong. I suppose I could have something like:

Value *getThreadLongMaybeUntagged(Value *&SlotPtr, Value *&ThreadLong) {
    if (!SlotPtr)
        SlotPtr = getHwasanThreadSlotPtr(IRB, IntptrTy);
    if (!ThreadLong)
        ThreadLong = IRB.CreateLoad(IntptrTy, SlotPtr);
    // Extract the address field from ThreadLong. Unnecessary on AArch64 with TBI.
    return TargetTriple.isAArch64() ? ThreadLong : untagPointer(IRB, ThreadLong);
}

  Value *ThreadLongMaybeUntagged = nullptr, *SlotPtr = nullptr, *ThreadLong = nullptr;
  if (WithFrameRecord) {
    if (ClRecordStackHistoryWithCalls) {
        ....
    } else {
        ThreadLongMaybeUntagged = getThreadLongMaybeUntagged(SlotPtr, ThreadLong);
        ....
    }
  }

  if (!ShadowBase) {
    if (!ThreadLongMaybeUntagged) {
      ThreadLongMaybeUntagged = getThreadLongMaybeUntagged(SlotPtr, ThreadLong);
      ...

where I can save intermediate values?

[vitalybuka]

LGTM

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

1147

you can add this function in a separate patch

1173–1225

switch/case

1228

function local lambda works for me

auto getThreadLongMaybeUntagged = [&]() {
    if (!SlotPtr)
        SlotPtr = getHwasanThreadSlotPtr(IRB, IntptrTy);
    if (!ThreadLong)
        ThreadLong = IRB.CreateLoad(IntptrTy, SlotPtr);
    // Extract the address field from ThreadLong. Unnecessary on AArch64 with TBI.
    return TargetTriple.isAArch64() ? ThreadLong : untagPointer(IRB, ThreadLong);
}

[leonardchan]

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

1147

Made https://reviews.llvm.org/D129315

[leonardchan]

ping @mcgrathr does this look good to you?

[mcgrathr]

LGTM, I didn't realize Phabricator considered me to be blocking.

compiler-rt/lib/hwasan/hwasan.cpp

 static const u8 kFallbackTag = 0xBB & kTagMask;
 
 u8 __hwasan_generate_tag() {
   Thread *t = GetCurrentThread();
   if (!t) return kFallbackTag;
   return t->GenerateRandomTag();
 }
 
+void __hwasan_record_frame_record(u64 frame_record_info) {
+  Thread *t = GetCurrentThread();
+  if (t)
+    t->stack_allocations()->push(frame_record_info);
+}
+
 #if !SANITIZER_SUPPORTS_WEAK_HOOKS
 extern "C" {
 SANITIZER_INTERFACE_ATTRIBUTE SANITIZER_WEAK_ATTRIBUTE
 const char* __hwasan_default_options() { return ""; }
 }  // extern "C"
 #endif
 
 extern "C" {

compiler-rt/lib/hwasan/hwasan_interface_internal.h

 
 SANITIZER_INTERFACE_ATTRIBUTE
 void __hwasan_thread_exit();
 
 SANITIZER_INTERFACE_ATTRIBUTE
 void __hwasan_print_memory_usage();
 
 SANITIZER_INTERFACE_ATTRIBUTE
+void __hwasan_record_frame_record(u64 frame_record_info);
+
+SANITIZER_INTERFACE_ATTRIBUTE
 void *__hwasan_memcpy(void *dst, const void *src, uptr size);
 SANITIZER_INTERFACE_ATTRIBUTE
 void *__hwasan_memset(void *s, int c, uptr n);
 SANITIZER_INTERFACE_ATTRIBUTE
 void *__hwasan_memmove(void *dest, const void *src, uptr n);
 
 SANITIZER_INTERFACE_ATTRIBUTE
 void __hwasan_set_error_report_callback(void (*callback)(const char *));
 }  // extern "C"
 
 #endif  // HWASAN_INTERFACE_INTERNAL_H

compiler-rt/test/hwasan/TestCases/deep-recursion.c

 // RUN: %clang_hwasan -O1 %s -o %t
 // RUN: %env_hwasan_opts=stack_history_size=1 not %run %t 2>&1 | FileCheck %s --check-prefix=D1
 // RUN: %env_hwasan_opts=stack_history_size=2 not %run %t 2>&1 | FileCheck %s --check-prefix=D2
 // RUN: %env_hwasan_opts=stack_history_size=3 not %run %t 2>&1 | FileCheck %s --check-prefix=D3
 // RUN: %env_hwasan_opts=stack_history_size=5 not %run %t 2>&1 | FileCheck %s --check-prefix=D5
 // RUN:                                       not %run %t 2>&1 | FileCheck %s --check-prefix=DEFAULT
 
+// Run the same tests as above, but using the __hwasan_record_frame_record libcall.
+// The output should be the exact same.
+// RUN: %clang_hwasan -O1 %s -o %t.libcall -mllvm -hwasan-record-stack-history-with-calls=1
+// RUN: %env_hwasan_opts=stack_history_size=1 not %run %t.libcall 2>&1 | FileCheck %s --check-prefix=D1
+// RUN: %env_hwasan_opts=stack_history_size=2 not %run %t.libcall 2>&1 | FileCheck %s --check-prefix=D2
+// RUN: %env_hwasan_opts=stack_history_size=3 not %run %t.libcall 2>&1 | FileCheck %s --check-prefix=D3
+// RUN: %env_hwasan_opts=stack_history_size=5 not %run %t.libcall 2>&1 | FileCheck %s --check-prefix=D5
+// RUN:                                       not %run %t.libcall 2>&1 | FileCheck %s --check-prefix=DEFAULT
+
 // REQUIRES: stable-runtime
 
 // Stack histories are currently not recorded on x86.
 // XFAIL: x86_64
 
 #include <stdlib.h>
 // At least -O1 is needed for this function to not have a stack frame on
 // AArch64.

compiler-rt/test/hwasan/TestCases/stack-history-length.c

 // RUN: %clang_hwasan -O1 %s -o %t
 // RUN: %env_hwasan_opts=stack_history_size=2048 not %run %t 2045 2>&1 | FileCheck %s --check-prefix=YES
 // RUN: %env_hwasan_opts=stack_history_size=2048 not %run %t 2047 2>&1 | FileCheck %s --check-prefix=NO
 
+// Run the same tests as above, but using the __hwasan_record_frame_record libcall.
+// The output should be the exact same.
+// RUN: %clang_hwasan -O1 %s -o %t.libcall -mllvm -hwasan-record-stack-history-with-calls=1
+// RUN: %env_hwasan_opts=stack_history_size=2048 not %run %t.libcall 2045 2>&1 | FileCheck %s --check-prefix=YES
+// RUN: %env_hwasan_opts=stack_history_size=2048 not %run %t.libcall 2047 2>&1 | FileCheck %s --check-prefix=NO
+
 // REQUIRES: stable-runtime
 
 // Stack histories are currently not recorded on x86.
 // XFAIL: x86_64
 
 #include <stdlib.h>
 
 void USE(void *x) { // pretend_to_do_something(void *x)

compiler-rt/test/hwasan/TestCases/stack-uar.c

 // Tests use-after-return detection and reporting.
 // RUN: %clang_hwasan -g %s -o %t && not %run %t 2>&1 | FileCheck %s
 // RUN: %clang_hwasan -g %s -o %t && not %env_hwasan_opts=symbolize=0 %run %t 2>&1 | FileCheck %s --check-prefix=NOSYM
 
+// Run the same test as above, but using the __hwasan_record_frame_record libcall.
+// The output should be the exact same.
+// RUN: %clang_hwasan -g %s -o %t.libcall -mllvm -hwasan-record-stack-history-with-calls=1 && not %env_hwasan_opts=symbolize=0 %run %t.libcall 2>&1 | FileCheck %s --check-prefix=NOSYM
+
 // REQUIRES: stable-runtime
 
 // Stack histories currently are not recorded on x86.
 // XFAIL: x86_64
 
 void USE(void *x) { // pretend_to_do_something(void *x)
   __asm__ __volatile__("" : : "r" (x) : "memory");
 }

compiler-rt/test/hwasan/TestCases/stack-uas.c

 // Tests use-after-scope detection and reporting.
 // RUN: %clang_hwasan -mllvm -hwasan-use-after-scope -g %s -o %t && not %run %t 2>&1 | FileCheck %s
 // RUN: %clang_hwasan -mllvm -hwasan-use-after-scope -g %s -o %t && not %env_hwasan_opts=symbolize=0 %run %t 2>&1 | FileCheck %s --check-prefix=NOSYM
 
 // RUN: %clang_hwasan -mllvm -hwasan-use-after-scope=false -g %s -o %t && %run %t 2>&1
 // Use after scope is turned off by default.
 // RUN: %clang_hwasan -g %s -o %t && %run %t 2>&1
 
 // RUN: %clang_hwasan -mllvm -hwasan-use-after-scope -g %s -o %t && not %run %t 2>&1 | FileCheck %s
 
+// Run the same test as above, but using the __hwasan_record_frame_record libcall.
+// The output should be the exact same.
+// RUN: %clang_hwasan -mllvm -hwasan-use-after-scope -mllvm -hwasan-record-stack-history-with-calls=1 -g %s -o %t.libcall && not %env_hwasan_opts=symbolize=0 %run %t.libcall 2>&1 | FileCheck %s --check-prefix=NOSYM
+
 // REQUIRES: stable-runtime
 
 // Stack histories currently are not recorded on x86.
 // XFAIL: x86_64
 
 void USE(void *x) { // pretend_to_do_something(void *x)
   __asm__ __volatile__(""
                        :

llvm/lib/Transforms/Instrumentation/HWAddressSanitizer.cpp

ClWithIfunc("hwasan-with-ifunc",
                 cl::Hidden, cl::init(false));
 
 static cl::opt<bool> ClWithTls(
     "hwasan-with-tls",
     cl::desc("Access dynamic shadow through an thread-local pointer on "
              "platforms that support this"),
     cl::Hidden, cl::init(true));
 
 static cl::opt<bool>
     ClRecordStackHistory("hwasan-record-stack-history",
                          cl::desc("Record stack frames with tagged allocations "
                                   "in a thread-local ring buffer"),
                          cl::Hidden, cl::init(true));
+
+static cl::opt<bool> ClRecordStackHistoryWithCalls(
+    "hwasan-record-stack-history-with-calls",
+    cl::desc("If recording stack frames with tagged allocations, record them "
+             "with runtime library calls"),
+    cl::Hidden, cl::init(false));
+
 static cl::opt<bool>
     ClInstrumentMemIntrinsics("hwasan-instrument-mem-intrinsics",
                               cl::desc("instrument memory intrinsics"),
                               cl::Hidden, cl::init(true));
 
 static cl::opt<bool>
     ClInstrumentLandingPads("hwasan-instrument-landing-pads",
                             cl::desc("instrument landing pads"), cl::Hidden,
public:
 
   void emitPrologue(IRBuilder<> &IRB, bool WithFrameRecord);
 
   void instrumentGlobal(GlobalVariable *GV, uint8_t Tag);
   void instrumentGlobals();
 
   Value *getPC(IRBuilder<> &IRB);
   Value *getSP(IRBuilder<> &IRB);
+  Value *getFrameRecordInfo(IRBuilder<> &IRB);
 
   void instrumentPersonalityFunctions();
 
 private:
   LLVMContext *C;
   Module &M;
   const StackSafetyGlobalInfo *SSI;
   Triple TargetTriple;
private:
 
   Function *HwasanCtorFunction;
 
   FunctionCallee HwasanMemoryAccessCallback[2][kNumberOfAccessSizes];
   FunctionCallee HwasanMemoryAccessCallbackSized[2];
 
   FunctionCallee HwasanTagMemoryFunc;
   FunctionCallee HwasanGenerateTagFunc;
+  FunctionCallee HwasanRecordFrameRecordFunc;
 
   Constant *ShadowGlobal;
 
   Value *ShadowBase = nullptr;
   Value *StackBaseTag = nullptr;
   Value *CachedSP = nullptr;
   GlobalValue *ThreadPtrGlobal = nullptr;
 };
for (size_t AccessIsWrite = 0; AccessIsWrite <= 1; AccessIsWrite++) {
     }
   }
 
   HwasanTagMemoryFunc = M.getOrInsertFunction(
       "__hwasan_tag_memory", IRB.getVoidTy(), Int8PtrTy, Int8Ty, IntptrTy);
   HwasanGenerateTagFunc =
       M.getOrInsertFunction("__hwasan_generate_tag", Int8Ty);
 
+  HwasanRecordFrameRecordFunc = M.getOrInsertFunction(
+      "__hwasan_record_frame_record", IRB.getVoidTy(), Int64Ty);
+
   ShadowGlobal = M.getOrInsertGlobal("__hwasan_shadow",
                                      ArrayType::get(IRB.getInt8Ty(), 0));
 
   const std::string MemIntrinCallbackPrefix =
       (CompileKernel && !ClKasanMemIntrinCallbackPrefix)
           ? std::string("")
           : ClMemoryAccessCallbackPrefix;
   HWAsanMemmove = M.getOrInsertFunction(MemIntrinCallbackPrefix + "memmove",
if (!CachedSP) {
     CachedSP = IRB.CreatePtrToInt(
         IRB.CreateCall(GetStackPointerFn,
                        {Constant::getNullValue(IRB.getInt32Ty())}),
         IntptrTy);
   }
   return CachedSP;
 }
 
+Value *HWAddressSanitizer::getFrameRecordInfo(IRBuilder<> &IRB) {
+  // Prepare ring buffer data.
+  Value *PC = getPC(IRB);
+  Value *SP = getSP(IRB);
+
+  // Mix SP and PC.
+  // Assumptions:
+  // PC is 0x0000PPPPPPPPPPPP  (48 bits are meaningful, others are zero)
+  // SP is 0xsssssssssssSSSS0  (4 lower bits are zero)
+  // We only really need ~20 lower non-zero bits (SSSS), so we mix like this:
+  //       0xSSSSPPPPPPPPPPPP
+  SP = IRB.CreateShl(SP, 44);
+  return IRB.CreateOr(PC, SP);
+}
+
 void HWAddressSanitizer::emitPrologue(IRBuilder<> &IRB, bool WithFrameRecord) {
   if (!Mapping.InTls)
     ShadowBase = getShadowNonTls(IRB);
   else if (!WithFrameRecord && TargetTriple.isAndroid())
     ShadowBase = getDynamicShadowIfunc(IRB);
 
   if (!WithFrameRecord && ShadowBase)
     return;
 
+  Value *ThreadLongMaybeUntagged = nullptr;
+  if (WithFrameRecord) {
+    if (ClRecordStackHistoryWithCalls) {
+      // Emit a runtime call into hwasan rather than emitting instructions for
+      // recording stack history.
+      Value *FrameRecordInfo = getFrameRecordInfo(IRB);
+      IRB.CreateCall(HwasanRecordFrameRecordFunc, {FrameRecordInfo});
+    } else {
       Value *SlotPtr = getHwasanThreadSlotPtr(IRB, IntptrTy);
       assert(SlotPtr);
 
       Value *ThreadLong = IRB.CreateLoad(IntptrTy, SlotPtr);
-  // Extract the address field from ThreadLong. Unnecessary on AArch64 with TBI.
-  Value *ThreadLongMaybeUntagged =
+      // Extract the address field from ThreadLong. Unnecessary on AArch64 with
+      // TBI.
+      ThreadLongMaybeUntagged =
           TargetTriple.isAArch64() ? ThreadLong : untagPointer(IRB, ThreadLong);
 
-  if (WithFrameRecord) {
       StackBaseTag = IRB.CreateAShr(ThreadLong, 3);
 
       // Prepare ring buffer data.
+      // TODO: The SP and PC mixing is done here rather than calling
+      // `getFrameRecordInfo` to preserve existing codegen for code not emitting
+      // libcalls. Once this change lands, we could come back and refactor this
+      // to use `getFrameRecordInfo` and update codegen tests.
       Value *PC = getPC(IRB);
       Value *SP = getSP(IRB);
 
       // Mix SP and PC.
       // Assumptions:
       // PC is 0x0000PPPPPPPPPPPP  (48 bits are meaningful, others are zero)
       // SP is 0xsssssssssssSSSS0  (4 lower bits are zero)
-    // We only really need ~20 lower non-zero bits (SSSS), so we mix like this:
+      // We only really need ~20 lower non-zero bits (SSSS), so we mix like
+      // this:
       //       0xSSSSPPPPPPPPPPPP
       SP = IRB.CreateShl(SP, 44);
 
       // Store data to ring buffer.
-    Value *RecordPtr =
-        IRB.CreateIntToPtr(ThreadLongMaybeUntagged, IntptrTy->getPointerTo(0));
+      Value *RecordPtr = IRB.CreateIntToPtr(ThreadLongMaybeUntagged,
+                                            IntptrTy->getPointerTo(0));
       IRB.CreateStore(IRB.CreateOr(PC, SP), RecordPtr);
 
       // Update the ring buffer. Top byte of ThreadLong defines the size of the
       // buffer in pages, it must be a power of two, and the start of the buffer
       // must be aligned by twice that much. Therefore wrap around of the ring
       // buffer is simply Addr &= ~((ThreadLong >> 56) << 12).
       // The use of AShr instead of LShr is due to
       //   https://bugs.llvm.org/show_bug.cgi?id=39030
       // Runtime library makes sure not to use the highest bit.
       Value *WrapMask = IRB.CreateXor(
           IRB.CreateShl(IRB.CreateAShr(ThreadLong, 56), 12, "", true, true),
           ConstantInt::get(IntptrTy, (uint64_t)-1));
       Value *ThreadLongNew = IRB.CreateAnd(
           IRB.CreateAdd(ThreadLong, ConstantInt::get(IntptrTy, 8)), WrapMask);
       IRB.CreateStore(ThreadLongNew, SlotPtr);
     }
+  }
 
   if (!ShadowBase) {
+    if (!ThreadLongMaybeUntagged) {
+      Value *SlotPtr = getHwasanThreadSlotPtr(IRB, IntptrTy);
+      assert(SlotPtr);
+
+      Value *ThreadLong = IRB.CreateLoad(IntptrTy, SlotPtr);
+      // Extract the address field from ThreadLong. Unnecessary on AArch64 with
+      // TBI.
+      ThreadLongMaybeUntagged =
+          TargetTriple.isAArch64() ? ThreadLong : untagPointer(IRB, ThreadLong);
+    }
+
     // Get shadow base address by aligning RecordPtr up.
     // Note: this is not correct if the pointer is already aligned.
     // Runtime library will make sure this never happens.
     ShadowBase = IRB.CreateAdd(
         IRB.CreateOr(
             ThreadLongMaybeUntagged,
             ConstantInt::get(IntptrTy, (1ULL << kShadowBaseAlignment) - 1)),
         ConstantInt::get(IntptrTy, 1), "hwasan.shadow");

llvm/test/Instrumentation/HWAddressSanitizer/prologue.ll

 ; Test -hwasan-with-ifunc flag.
 ;
 ; RUN: opt -passes=hwasan -S < %s | \
-; RUN:     FileCheck %s --check-prefixes=CHECK,CHECK-NOGLOBAL,CHECK-TLS-SLOT,CHECK-HISTORY,CHECK-HISTORY-TLS-SLOT
+; RUN:     FileCheck %s --check-prefixes=CHECK,CHECK-NOGLOBAL,CHECK-TLS-SLOT,CHECK-HISTORY,CHECK-HISTORY-TLS-SLOT,CHECK-HISTORY-TLS
 ; RUN: opt -passes=hwasan -S -hwasan-with-ifunc=0 -hwasan-with-tls=1 -hwasan-record-stack-history=1 < %s | \
-; RUN:     FileCheck %s --check-prefixes=CHECK,CHECK-NOGLOBAL,CHECK-TLS-SLOT,CHECK-HISTORY,CHECK-HISTORY-TLS-SLOT
+; RUN:     FileCheck %s --check-prefixes=CHECK,CHECK-NOGLOBAL,CHECK-TLS-SLOT,CHECK-HISTORY,CHECK-HISTORY-TLS-SLOT,CHECK-HISTORY-TLS
 ; RUN: opt -passes=hwasan -S -hwasan-with-ifunc=0 -hwasan-with-tls=1 -hwasan-record-stack-history=0 < %s | \
 ; RUN:     FileCheck %s --check-prefixes=CHECK,CHECK-NOGLOBAL,CHECK-IFUNC,CHECK-NOHISTORY
 ; RUN: opt -passes=hwasan -S -hwasan-with-ifunc=0 -hwasan-with-tls=0 < %s | \
 ; RUN:     FileCheck %s --check-prefixes=CHECK,CHECK-GLOBAL,CHECK-NOHISTORY
 ; RUN: opt -passes=hwasan -S -hwasan-with-ifunc=1  -hwasan-with-tls=0 < %s | \
 ; RUN:     FileCheck %s --check-prefixes=CHECK,CHECK-IFUNC,CHECK-NOHISTORY
 ; RUN: opt -passes=hwasan -S -mtriple=aarch64-fuchsia < %s | \
-; RUN:     FileCheck %s --check-prefixes=CHECK,CHECK-ZERO-OFFSET,CHECK-SHORT-GRANULES,CHECK-HISTORY,CHECK-HWASAN-TLS,CHECK-HISTORY-HWASAN-TLS
+; RUN:     FileCheck %s --check-prefixes=CHECK,CHECK-ZERO-OFFSET,CHECK-SHORT-GRANULES,CHECK-HISTORY,CHECK-HWASAN-TLS,CHECK-HISTORY-HWASAN-TLS,CHECK-HISTORY-TLS
+; RUN: opt -passes=hwasan -S -mtriple=aarch64-fuchsia -hwasan-record-stack-history-with-calls=1 < %s | \
+; RUN:     FileCheck %s --check-prefixes=CHECK,CHECK-HISTORY,CHECK-HISTORY-LIBCALL
 
 target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
 target triple = "aarch64--linux-android22"
 
 ; CHECK-IFUNC: @__hwasan_shadow = external global [0 x i8]
 ; CHECK-NOIFUNC: @__hwasan_shadow_memory_dynamic_address = external global i64
 
 define i32 @test_load(i32* %a) sanitize_hwaddress {

 ; CHECK-TLS-SLOT:   %[[C:[^ ]*]] = bitcast i8* %[[B]] to i64*
 ; CHECK-TLS-SLOT:   %[[D:[^ ]*]] = load i64, i64* %[[C]]
 ; CHECK-TLS-SLOT:   %[[E:[^ ]*]] = ashr i64 %[[D]], 3
 ; CHECK-HWASAN-TLS: %[[D:[^ ]*]] = load i64, i64* @__hwasan_tls, align 8
 ; CHECK-HWASAN-TLS: %[[E:[^ ]*]] = ashr i64 %[[D]], 3
 
 ; CHECK-NOHISTORY-NOT: store i64
 
-; CHECK-HISTORY: call i64 @llvm.read_register.i64(metadata [[MD:![0-9]*]])
-; CHECK-HISTORY: %[[PTR:[^ ]*]] = inttoptr i64 %[[D]] to i64*
-; CHECK-HISTORY: store i64 %{{.*}}, i64* %[[PTR]]
-; CHECK-HISTORY: %[[D1:[^ ]*]] = ashr i64 %[[D]], 56
-; CHECK-HISTORY: %[[D2:[^ ]*]] = shl nuw nsw i64 %[[D1]], 12
-; CHECK-HISTORY: %[[D3:[^ ]*]] = xor i64 %[[D2]], -1
-; CHECK-HISTORY: %[[D4:[^ ]*]] = add i64 %[[D]], 8
-; CHECK-HISTORY: %[[D5:[^ ]*]] = and i64 %[[D4]], %[[D3]]
+; When watching stack history, all code paths attempt to get PC and SP and mix them together.
+; CHECK-HISTORY: %[[PC:[^ ]*]] = call i64 @llvm.read_register.i64(metadata [[MD:![0-9]*]])
+; CHECK-HISTORY: %[[SP0:[^ ]*]] = call i8* @llvm.frameaddress.p0i8(i32 0)
+; CHECK-HISTORY: %[[SP1:[^ ]*]] = ptrtoint i8* %[[SP0]] to i64
+; CHECK-HISTORY: %[[SP2:[^ ]*]] = shl i64 %[[SP1]], 44
+
+; CHECK-HISTORY-TLS: %[[PTR:[^ ]*]] = inttoptr i64 %[[D]] to i64*
+; CHECK-HISTORY: %[[MIX:[^ ]*]] = or i64 %[[PC]], %[[SP2]]
+; CHECK-HISTORY-TLS: store i64 %[[MIX]], i64* %[[PTR]]
+; CHECK-HISTORY-TLS: %[[D1:[^ ]*]] = ashr i64 %[[D]], 56
+; CHECK-HISTORY-TLS: %[[D2:[^ ]*]] = shl nuw nsw i64 %[[D1]], 12
+; CHECK-HISTORY-TLS: %[[D3:[^ ]*]] = xor i64 %[[D2]], -1
+; CHECK-HISTORY-TLS: %[[D4:[^ ]*]] = add i64 %[[D]], 8
+; CHECK-HISTORY-TLS: %[[D5:[^ ]*]] = and i64 %[[D4]], %[[D3]]
 ; CHECK-HISTORY-TLS-SLOT: store i64 %[[D5]], i64* %[[C]]
 ; CHECK-HISTORY-HWASAN-TLS: store i64 %[[D5]], i64* @__hwasan_tls
+; CHECK-HISTORY-LIBCALL: call void @__hwasan_record_frame_record(i64 %[[MIX]])
 
 ; CHECK-TLS:   %[[F:[^ ]*]] = or i64 %[[D]], 4294967295
 ; CHECK-TLS:   = add i64 %[[F]], 1
 
+; CHECK-HISTORY-LIBCALL: %[[E:hwasan.stack.base.tag]] = xor
 ; CHECK-HISTORY: = xor i64 %[[E]], 0
 
 ; CHECK-NOHISTORY-NOT: store i64
 
 
 entry:
   %x = alloca i32, align 4
   call void @use(i32* %x)
   ret void
 }
 
 ; CHECK-HISTORY: [[MD]] = !{!"pc"}