[Asan] Add "funclet" OpBundle to Asan calls that are generated inside a funclet

[saudi]

Summary

This patch implements a fix for issue https://github.com/llvm/llvm-project/issues/64990, implementing the solution suggested in the comments of issue https://github.com/llvm/llvm-project/issues/39667

It ensures that when asan creates calls (__asan_*) inside of a funclet, those calls are tagged with the required "funclet" OpBundle.
It only applies to EH personality that use funclets, especially Windows EH.

Also, changed localescape test, switching its EH personality to match the code which doesn't match the funclet scoping.

[rnk]

Seeking a code reviewer, +@aeubanks, @vitalybuka

This came up in https://github.com/google/sanitizers/issues/749

The general approach makes sense to me.

[aeubanks]

can you update the description to say https://github.com/llvm/llvm-project/issues/39667 (or #39667) instead of PR39667? llvm.org/PR39667 redirects to a different issue

[saudi]

Rebased on main.

Also added a check to allow colorless BasicBlocks from colorEHFunclets(), for the case of unreachable BB, which may happen during the asan pass.

[aeubanks]

rnk explained the background for this to me but I'm still somewhat unfamiliar with the details so you may need to explain some things to me

it'd be nice to add an end-to-end test in compiler-rt for whatever case you were seeing

the test coverage doesn't seem sufficient, can you add some more tests?

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

649

I'd separate out a bool instead of using std::optional, that's a bit clearer

702

should run clang-format before uploading

706

I'm unfamiliar with this code, why is this true?

2726–2729

is EH relevant to the ctor/dtor functions?

llvm/test/Instrumentation/AddressSanitizer/asan-funclet.ll

4

please use llvm/utils/update_test_checks.py instead of manual CHECK lines

7

test needs REQUIRES: x86-registered-target

22

please remove irrelevant things in the IR like dso_local/noundef/etc and also make the function names a little nicer

[saudi]

Updated to address comments.

• • Changed std::optional to use a bool instead
  • Removed changes for __asan_* calls that don't apply to Windows EH handling scenarios (module ctor/dtor, globals array)
• Added compiler-rt test
• Improved the test for more coverage

I didn't update the test with llvm/utils/update_test_checks.py, I have more questions about it, which I will post soon.

[saudi]

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

706

This is a constraint on the funclets in the Windows EH model.

An illustration in code is in WinEHPrepare pass verifyPreparedFunclets method here : https://github.com/llvm/llvm-project/blob/llvmorg-18-init/llvm/lib/CodeGen/WinEHPrepare.cpp#L1167
(from which I copied the assert)

In the documentation I found this: https://llvm.org/docs/ExceptionHandling.html#funclet-parent-tokens
It explains that the funclet structure must represent a tree.

I'm not an expert in this, but my understanding is that in WinEH model, the funclets need to be structured as a tree; the coloring allows to identify which "node" each EH-related BasicBlock belongs to, having the constraint that the BB must belong to one and only one node.

@rnk can you confirm?

2726–2729

My approach was to play it safe by using the RuntimeCallInserter for any runtime calls to __asan_*() introduced by the AddressSanitizer pass; the RuntimeCallInserter being responsible for adding funclet bundles as a postprocess.

I agree it's confusing. I updated the code where I removed the modifications for unneeded cases, where the __asan_xxx calls can't be in a EH pad, like this example of the module ctor/dtor

llvm/test/Instrumentation/AddressSanitizer/asan-funclet.ll

4

The pass introduces a lot of instructions and a few basic blocks. The output of update_test_checks.py gets large and too specific for testing this particular feature. It would easily break when asan gets modified in the future.

This is my first time using that script. Is the usual approach to apply update_test_checks.py and only keep the CHECK lines necessary with potential minor modifications?

[aeubanks]

mostly lgtm, but one major suggestion

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

663

these initialize/reset aren't great in terms of keeping C++ lifetimes straight, I think we should go with a more RAII approach where we construct a RuntimeCallInserter per function we instrument and pass that around. this will make sure we automatically handle all the corner cases like the LooksLikeCodeInBug11395 codepath, and also make sure we never reuse RuntimeCallInserter for multiple functions. then we can get rid of all the manual calls to initialize/finalize.

llvm/test/Instrumentation/AddressSanitizer/asan-funclet.ll

4

typically yeah we still recommend people to use update_test_checks.py since updating them is easy if other patches change the tests, and manual checks are error-prone and can often become obsolete. but most asan tests aren't using it so this is fine

[saudi]

Applied suggestion in comment (making RuntimeCallInserter use an RAII approach)

[vitalybuka]

llvm/test/Instrumentation/AddressSanitizer/asan-funclet.ll

4

Can you please precommit the test generated with update_test_checks without your patch
so in this patch we can see a difference in generated code.
Assuming compiler does not crash without the patch.

[rnk]

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

706

I think it is possible to construct cases where valid transforms will lead to multi-color blocks. Consider patches like https://reviews.llvm.org/D29428

I would actually strengthen the assert here to report an error via the LLVMContext. That seems to be the behavior that folks want.

[saudi]

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

706

If I understand correctly:

• WinEHPrepare pass is run later, and ensures the blocks are monochromatic
• In AddressSanitizer pass, this constraint is not guaranteed, we could be trying to introduce a runtime call inside a multi-color block

And you are suggesting detecting and reporting that case as a remark.

Could another approach be to tag the new calls somehow, in a way that would let WinHEPrepare know that instead of discarding it, it should just add the missing OpBundle?

And if so, could you point me in the right direction? (I was thinking of adding a new opbundle "autofunclet", but there might be much better ways?)

llvm/test/Instrumentation/AddressSanitizer/asan-funclet.ll

4

Just to be sure, are you suggesting that I do the following?

1. prepare the file asan-funclet.ll before the fix is applied, apply update_test_checks.py and do a NFC commit with this single file asan-funclet.ll
2. apply my code change, and update asan-funclet.ll using update_test_checks.py -> update this patch, so that we can see more easily what happens?

Note that the test file will become much larger: applying update_test_checks.py generates 500+ lines of checks. It's why I went with the more manual checks, but I can surely do that.

compiler-rt/test/asan/TestCases/Windows/issue64990.cpp

+// Repro for the issue #64990: Asan with Windows EH generates __asan_xxx runtime calls without required funclet tokens
+// RUN: %clang_cl_asan %Od %s -EHsc %Fe%t
+// RUN: not %run %t 2>&1 | FileCheck %s
+
+char buff1[6] = "hello";
+char buff2[6] = "hello";
+
+int main(int argc, char **argv) {
+  try {
+    throw 1;
+  } catch (...) {
+    // Make asan generate call to __asan_memcpy inside the EH pad.
+    __builtin_memcpy(buff1, buff2 + 3, 6);
+  }
+  return 0;
+}
+
+// CHECK: SUMMARY: AddressSanitizer: global-buffer-overflow {{.*}} in __asan_memcpy

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

 #include "llvm/IR/Comdat.h"
 #include "llvm/IR/Constant.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DIBuilder.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/DebugInfoMetadata.h"
 #include "llvm/IR/DebugLoc.h"
 #include "llvm/IR/DerivedTypes.h"
+#include "llvm/IR/EHPersonalities.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/GlobalAlias.h"
 #include "llvm/IR/GlobalValue.h"
 #include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/InlineAsm.h"
 #include "llvm/IR/InstVisitor.h"
 #include "llvm/IR/InstrTypes.h"
static uint64_t GetCtorAndDtorPriority(Triple &TargetTriple) {
   if (TargetTriple.isOSEmscripten()) {
     return kAsanEmscriptenCtorAndDtorPriority;
   } else {
     return kAsanCtorAndDtorPriority;
   }
 }
 
 namespace {
+/// Helper class to keep track of the inserted runtime calls during a pass on a
+/// single Function. At the end of that pass, calling finalize() will detect and
+/// add the corresponding funclet OpBundle to the inserted calls.
+class RuntimeCallInserter {
+  Function *OwnerFn = nullptr;
+  bool TrackInsertedCalls = false;
+  std::vector<CallInst *> InsertedCalls;
+
+  void reset() {
+    OwnerFn = nullptr;
+    TrackInsertedCalls = false;
+    InsertedCalls.clear();
+  }
+
+public:
+  ~RuntimeCallInserter() { assert(!isInitialized()); }
+
+  bool isInitialized() const { return OwnerFn != nullptr; }
+
+  void initialize(Function &Fn) {
+    assert(!isInitialized() && !TrackInsertedCalls);
+    OwnerFn = &Fn;
+    if (Fn.hasPersonalityFn()) {
+      auto Personality = classifyEHPersonality(Fn.getPersonalityFn());
+      if (isScopedEHPersonality(Personality))
+        TrackInsertedCalls = true;
+    }
+  }
+
+  CallInst *createRuntimeCall(IRBuilder<> &IRB, FunctionCallee Callee,
+                              ArrayRef<Value *> Args = {},
+                              const Twine &Name = "") {
+    assert(isInitialized());
+    assert(IRB.GetInsertBlock()->getParent() == OwnerFn);
+
+    CallInst *Inst = IRB.CreateCall(Callee, Args, Name, nullptr);
+    if (TrackInsertedCalls) {
+      auto Fn = dyn_cast<Function>(Callee.getCallee());
+      bool IsIntrinsicCallee = Fn && Fn->isIntrinsic();
+      if (!IsIntrinsicCallee)
+        InsertedCalls.push_back(Inst);
+    }
+    return Inst;
+  }
+
+  void finalize() {
+    assert(isInitialized());
+    if (!TrackInsertedCalls || InsertedCalls.empty()) {
+      reset();
+      return;
+    }
+
+    DenseMap<BasicBlock *, ColorVector> BlockColors = colorEHFunclets(*OwnerFn);
+    for (CallInst *CI : InsertedCalls) {
+      BasicBlock *BB = CI->getParent();
+      assert(BB && "Instruction doesn't belong to a BasicBlock");
+      assert(BB->getParent() == OwnerFn &&
+             "Instruction doesn't belong to the expected Function!");
+
+      ColorVector &Colors = BlockColors[BB];
+      // A BB could be colorless when it is unreachable, in which case it will
+      // be removed by a later pass anyway.
+      if (Colors.empty())
+        continue;
+      assert(Colors.size() == 1 && "Expected monochromatic BB!");
+
+      BasicBlock *Color = Colors.front();
+      Instruction *EHPad = Color->getFirstNonPHI();
+
+      if (EHPad && EHPad->isEHPad()) {
+        // Replace CI with a clone with funclet OperandBundle
+        OperandBundleDef OB("funclet", EHPad);
+        auto *NewCall =
+            CallBase::addOperandBundle(CI, LLVMContext::OB_funclet, OB, CI);
+        NewCall->copyMetadata(*CI);
+        CI->replaceAllUsesWith(NewCall);
+        CI->eraseFromParent();
+      }
+    }
+    reset();
+  }
+};
 
 /// AddressSanitizer: instrument the code in module to find memory bugs.
 struct AddressSanitizer {
   AddressSanitizer(Module &M, const StackSafetyGlobalInfo *SSGI,
                    bool CompileKernel = false, bool Recover = false,
                    bool UseAfterScope = false,
                    AsanDetectStackUseAfterReturnMode UseAfterReturn =
                        AsanDetectStackUseAfterReturnMode::Runtime)
struct AddressSanitizer {
   void instrumentMemIntrinsic(MemIntrinsic *MI);
   Value *memToShadow(Value *Shadow, IRBuilder<> &IRB);
   bool suppressInstrumentationSiteForDebug(int &Instrumented);
   bool instrumentFunction(Function &F, const TargetLibraryInfo *TLI);
   bool maybeInsertAsanInitAtFunctionEntry(Function &F);
   bool maybeInsertDynamicShadowAtFunctionEntry(Function &F);
   void markEscapedLocalAllocas(Function &F);
 
+  CallInst *createRuntimeCall(IRBuilder<> &IRB, FunctionCallee Callee,
+                              ArrayRef<Value *> Args = {},
+                              const Twine &Name = "") {
+    return RtCallInserter.createRuntimeCall(IRB, Callee, Args, Name);
+  }
+
 private:
   friend struct FunctionStackPoisoner;
 
   void initializeCallbacks(Module &M, const TargetLibraryInfo *TLI);
 
   bool LooksLikeCodeInBug11395(Instruction *I);
   bool GlobalIsLinkerInitialized(GlobalVariable *G);
   bool isSafeAccess(ObjectSizeOffsetVisitor &ObjSizeVis, Value *Addr,
                     TypeSize TypeStoreSize) const;
 
   /// Helper to cleanup per-function state.
   struct FunctionStateRAII {
     AddressSanitizer *Pass;
 
     FunctionStateRAII(AddressSanitizer *Pass) : Pass(Pass) {
       assert(Pass->ProcessedAllocas.empty() &&
              "last pass forgot to clear cache");
       assert(!Pass->LocalDynamicShadow);
     }
 
     ~FunctionStateRAII() {
       Pass->LocalDynamicShadow = nullptr;
       Pass->ProcessedAllocas.clear();
+      assert(!Pass->RtCallInserter.isInitialized());
     }
   };
 
   LLVMContext *C;
   const DataLayout *DL;
   Triple TargetTriple;
   int LongSize;
   bool CompileKernel;
   bool Recover;
   bool UseAfterScope;
   AsanDetectStackUseAfterReturnMode UseAfterReturn;
   Type *IntptrTy;
   Type *Int8PtrTy;
   Type *Int32Ty;
   ShadowMapping Mapping;
   FunctionCallee AsanHandleNoReturnFunc;
   FunctionCallee AsanPtrCmpFunction, AsanPtrSubFunction;
   Constant *AsanShadowGlobal;
 
+  RuntimeCallInserter RtCallInserter;
+
   // These arrays is indexed by AccessIsWrite, Experiment and log2(AccessSize).
   FunctionCallee AsanErrorCallback[2][2][kNumberOfAccessSizes];
   FunctionCallee AsanMemoryAccessCallback[2][2][kNumberOfAccessSizes];
 
   // These arrays is indexed by AccessIsWrite and Experiment.
   FunctionCallee AsanErrorCallbackSized[2][2];
   FunctionCallee AsanMemoryAccessCallbackSized[2][2];
 
if (!isa<ReturnInst>(InstBefore)) {
           {IntptrTy});
 
       Value *DynamicAreaOffset = IRB.CreateCall(DynamicAreaOffsetFunc, {});
 
       DynamicAreaPtr = IRB.CreateAdd(IRB.CreatePtrToInt(SavedStack, IntptrTy),
                                      DynamicAreaOffset);
     }
 
-    IRB.CreateCall(
-        AsanAllocasUnpoisonFunc,
+    ASan.createRuntimeCall(
+        IRB, AsanAllocasUnpoisonFunc,
         {IRB.CreateLoad(IntptrTy, DynamicAllocaLayout), DynamicAreaPtr});
   }
 
   // Unpoison dynamic allocas redzones.
   void unpoisonDynamicAllocas() {
     for (Instruction *Ret : RetVec)
       unpoisonDynamicAllocasBeforeInst(Ret, DynamicAllocaLayout);
 
Value *AddressSanitizer::memToShadow(Value *Shadow, IRBuilder<> &IRB) {
   else
     return IRB.CreateAdd(Shadow, ShadowBase);
 }
 
 // Instrument memset/memmove/memcpy
 void AddressSanitizer::instrumentMemIntrinsic(MemIntrinsic *MI) {
   InstrumentationIRBuilder IRB(MI);
   if (isa<MemTransferInst>(MI)) {
-    IRB.CreateCall(
-        isa<MemMoveInst>(MI) ? AsanMemmove : AsanMemcpy,
+    createRuntimeCall(
+        IRB, isa<MemMoveInst>(MI) ? AsanMemmove : AsanMemcpy,
         {IRB.CreatePointerCast(MI->getOperand(0), IRB.getInt8PtrTy()),
          IRB.CreatePointerCast(MI->getOperand(1), IRB.getInt8PtrTy()),
          IRB.CreateIntCast(MI->getOperand(2), IntptrTy, false)});
   } else if (isa<MemSetInst>(MI)) {
-    IRB.CreateCall(
-        AsanMemset,
+    createRuntimeCall(
+        IRB, AsanMemset,
         {IRB.CreatePointerCast(MI->getOperand(0), IRB.getInt8PtrTy()),
          IRB.CreateIntCast(MI->getOperand(1), IRB.getInt32Ty(), false),
          IRB.CreateIntCast(MI->getOperand(2), IntptrTy, false)});
   }
   MI->eraseFromParent();
 }
 
 /// Check if we want (and can) handle this alloca.
void AddressSanitizer::instrumentPointerComparisonOrSubtraction(
     Instruction *I) {
   IRBuilder<> IRB(I);
   FunctionCallee F = isa<ICmpInst>(I) ? AsanPtrCmpFunction : AsanPtrSubFunction;
   Value *Param[2] = {I->getOperand(0), I->getOperand(1)};
   for (Value *&i : Param) {
     if (i->getType()->isPointerTy())
       i = IRB.CreatePointerCast(i, IntptrTy);
   }
-  IRB.CreateCall(F, Param);
+  createRuntimeCall(IRB, F, Param);
 }
 
 static void doInstrumentAddress(AddressSanitizer *Pass, Instruction *I,
                                 Instruction *InsertBefore, Value *Addr,
                                 MaybeAlign Alignment, unsigned Granularity,
                                 TypeSize TypeStoreSize, bool IsWrite,
                                 Value *SizeArgument, bool UseCalls,
                                 uint32_t Exp) {
Instruction *AddressSanitizer::generateCrashCode(Instruction *InsertBefore,
                                                  size_t AccessSizeIndex,
                                                  Value *SizeArgument,
                                                  uint32_t Exp) {
   InstrumentationIRBuilder IRB(InsertBefore);
   Value *ExpVal = Exp == 0 ? nullptr : ConstantInt::get(IRB.getInt32Ty(), Exp);
   CallInst *Call = nullptr;
   if (SizeArgument) {
     if (Exp == 0)
-      Call = IRB.CreateCall(AsanErrorCallbackSized[IsWrite][0],
+      Call = createRuntimeCall(IRB, AsanErrorCallbackSized[IsWrite][0],
                                {Addr, SizeArgument});
     else
-      Call = IRB.CreateCall(AsanErrorCallbackSized[IsWrite][1],
+      Call = createRuntimeCall(IRB, AsanErrorCallbackSized[IsWrite][1],
                                {Addr, SizeArgument, ExpVal});
   } else {
     if (Exp == 0)
-      Call =
-          IRB.CreateCall(AsanErrorCallback[IsWrite][0][AccessSizeIndex], Addr);
+      Call = createRuntimeCall(
+          IRB, AsanErrorCallback[IsWrite][0][AccessSizeIndex], Addr);
     else
-      Call = IRB.CreateCall(AsanErrorCallback[IsWrite][1][AccessSizeIndex],
-                            {Addr, ExpVal});
+      Call = createRuntimeCall(
+          IRB, AsanErrorCallback[IsWrite][1][AccessSizeIndex], {Addr, ExpVal});
   }
 
   Call->setCannotMerge();
   return Call;
 }
 
 Value *AddressSanitizer::createSlowPathCmp(IRBuilder<> &IRB, Value *AddrLong,
                                            Value *ShadowValue,
IRB.CreateCall(
         {IRB.CreatePointerCast(Addr, Int8PtrTy),
          ConstantInt::get(Int32Ty, AccessInfo.Packed)});
     return;
   }
 
   Value *AddrLong = IRB.CreatePointerCast(Addr, IntptrTy);
   if (UseCalls) {
     if (Exp == 0)
-      IRB.CreateCall(AsanMemoryAccessCallback[IsWrite][0][AccessSizeIndex],
-                     AddrLong);
+      createRuntimeCall(
+          IRB, AsanMemoryAccessCallback[IsWrite][0][AccessSizeIndex], AddrLong);
     else
-      IRB.CreateCall(AsanMemoryAccessCallback[IsWrite][1][AccessSizeIndex],
+      createRuntimeCall(IRB,
+                        AsanMemoryAccessCallback[IsWrite][1][AccessSizeIndex],
                         {AddrLong, ConstantInt::get(IRB.getInt32Ty(), Exp)});
     return;
   }
 
   Type *ShadowTy =
       IntegerType::get(*C, std::max(8U, TypeStoreSize >> Mapping.Scale));
   Type *ShadowPtrTy = PointerType::get(ShadowTy, 0);
   Value *ShadowPtr = memToShadow(AddrLong, IRB);
   const uint64_t ShadowAlign =
void AddressSanitizer::instrumentUnusualSizeOrAlignment(
     bool IsWrite, Value *SizeArgument, bool UseCalls, uint32_t Exp) {
   InstrumentationIRBuilder IRB(InsertBefore);
   Value *NumBits = IRB.CreateTypeSize(IntptrTy, TypeStoreSize);
   Value *Size = IRB.CreateLShr(NumBits, ConstantInt::get(IntptrTy, 3));
 
   Value *AddrLong = IRB.CreatePointerCast(Addr, IntptrTy);
   if (UseCalls) {
     if (Exp == 0)
-      IRB.CreateCall(AsanMemoryAccessCallbackSized[IsWrite][0],
+      createRuntimeCall(IRB, AsanMemoryAccessCallbackSized[IsWrite][0],
                         {AddrLong, Size});
     else
-      IRB.CreateCall(AsanMemoryAccessCallbackSized[IsWrite][1],
-                     {AddrLong, Size, ConstantInt::get(IRB.getInt32Ty(), Exp)});
+      createRuntimeCall(
+          IRB, AsanMemoryAccessCallbackSized[IsWrite][1],
   } else {
     Value *SizeMinusOne = IRB.CreateSub(Size, ConstantInt::get(IntptrTy, 1));
     Value *LastByte = IRB.CreateIntToPtr(
         IRB.CreateAdd(AddrLong, SizeMinusOne),
         Addr->getType());
     instrumentAddress(I, InsertBefore, Addr, {}, 8, IsWrite, Size, false, Exp);
     instrumentAddress(I, InsertBefore, LastByte, {}, 8, IsWrite, Size, false,
                       Exp);
if (UseCtorComdat && TargetTriple.isOSBinFormatELF() && CtorComdat) {
     }
   } else {
     if (AsanCtorFunction)
       appendToGlobalCtors(M, AsanCtorFunction, Priority);
     if (AsanDtorFunction)
       appendToGlobalDtors(M, AsanDtorFunction, Priority);
   }
 
   return true;
 }
 
 void AddressSanitizer::initializeCallbacks(Module &M, const TargetLibraryInfo *TLI) {
   IRBuilder<> IRB(*C);
   // Create __asan_report* callbacks.
   // IsWrite, TypeSize and Exp are encoded in the function name.
   for (int Exp = 0; Exp < 2; Exp++) {
     for (size_t AccessIsWrite = 0; AccessIsWrite <= 1; AccessIsWrite++) {
       const std::string TypeStr = AccessIsWrite ? "store" : "load";
       const std::string ExpStr = Exp ? "exp_" : "";
       const std::string EndingStr = Recover ? "_noabort" : "";
if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation))
     return FunctionModified;
 
   LLVM_DEBUG(dbgs() << "ASAN instrumenting:\n" << F << "\n");
 
   initializeCallbacks(*F.getParent(), TLI);
 
   FunctionStateRAII CleanupObj(this);
 
+  RtCallInserter.initialize(F);
+
   FunctionModified |= maybeInsertDynamicShadowAtFunctionEntry(F);
 
   // We can't instrument allocas used with llvm.localescape. Only static allocas
   // can be passed to that intrinsic.
   markEscapedLocalAllocas(F);
 
   // We want to instrument every address only once per basic block (unless there
   // are calls between uses).
   SmallPtrSet<Value *, 16> TempsToInstrument;
   SmallVector<InterestingMemoryOperand, 16> OperandsToInstrument;
   SmallVector<MemIntrinsic *, 16> IntrinToInstrument;
   SmallVector<Instruction *, 8> NoReturnCalls;
   SmallVector<BasicBlock *, 16> AllBlocks;
   SmallVector<Instruction *, 16> PointerComparisonsOrSubtracts;
 
   // Fill the set of memory operations to instrument.
   for (auto &BB : F) {
     AllBlocks.push_back(&BB);
     TempsToInstrument.clear();
     int NumInsnsPerBB = 0;
     for (auto &Inst : BB) {
-      if (LooksLikeCodeInBug11395(&Inst)) return false;
+      if (LooksLikeCodeInBug11395(&Inst)) {
+        RtCallInserter.finalize();
+        return false;
+      }
       // Skip instructions inserted by another instrumentation.
       if (Inst.hasMetadata(LLVMContext::MD_nosanitize))
         continue;
       SmallVector<InterestingMemoryOperand, 1> InterestingOperands;
       getInterestingMemoryOperands(&Inst, InterestingOperands);
 
       if (!InterestingOperands.empty()) {
         for (auto &Operand : InterestingOperands) {
bool AddressSanitizer::instrumentFunction(Function &F,
 
   FunctionStackPoisoner FSP(F, *this);
   bool ChangedStack = FSP.runOnFunction();
 
   // We must unpoison the stack before NoReturn calls (throw, _exit, etc).
   // See e.g. https://github.com/google/sanitizers/issues/37
   for (auto *CI : NoReturnCalls) {
     IRBuilder<> IRB(CI);
-    IRB.CreateCall(AsanHandleNoReturnFunc, {});
+    createRuntimeCall(IRB, AsanHandleNoReturnFunc, {});
   }
 
   for (auto *Inst : PointerComparisonsOrSubtracts) {
     instrumentPointerComparisonOrSubtraction(Inst);
     FunctionModified = true;
   }
 
   if (ChangedStack || !NoReturnCalls.empty())
     FunctionModified = true;
 
+  RtCallInserter.finalize();
+
   LLVM_DEBUG(dbgs() << "ASAN done instrumenting: " << FunctionModified << " "
                     << F << "\n");
 
   return FunctionModified;
 }
 
 // Workaround for bug 11395: we don't want to instrument stack in functions
 // with large assembly blobs (32-bit only), otherwise reg alloc may crash.
if (!AsanSetShadowFunc[Val])
       continue;
 
     // Skip same values.
     for (; j < End && ShadowMask[j] && Val == ShadowBytes[j]; ++j) {
     }
 
     if (j - i >= ClMaxInlinePoisoningSize) {
       copyToShadowInline(ShadowMask, ShadowBytes, Done, i, IRB, ShadowBase);
-      IRB.CreateCall(AsanSetShadowFunc[Val],
-                     {IRB.CreateAdd(ShadowBase, ConstantInt::get(IntptrTy, i)),
-                      ConstantInt::get(IntptrTy, j - i)});
+      ASan.createRuntimeCall(
+          IRB, AsanSetShadowFunc[Val],
       Done = j;
     }
   }
 
   copyToShadowInline(ShadowMask, ShadowBytes, Done, End, IRB, ShadowBase);
 }
 
 // Fake stack allocator (asan_fake_stack.h) has 11 size classes
if (ASan.UseAfterReturn == AsanDetectStackUseAfterReturnMode::Runtime) {
           IRB.CreateLoad(IRB.getInt32Ty(), OptionDetectUseAfterReturn),
           Constant::getNullValue(IRB.getInt32Ty()));
       Instruction *Term =
           SplitBlockAndInsertIfThen(UseAfterReturnIsEnabled, InsBefore, false);
       IRBuilder<> IRBIf(Term);
       StackMallocIdx = StackMallocSizeClass(LocalStackSize);
       assert(StackMallocIdx <= kMaxAsanStackMallocSizeClass);
       Value *FakeStackValue =
-          IRBIf.CreateCall(AsanStackMallocFunc[StackMallocIdx],
+          ASan.createRuntimeCall(IRBIf, AsanStackMallocFunc[StackMallocIdx],
                                  ConstantInt::get(IntptrTy, LocalStackSize));
       IRB.SetInsertPoint(InsBefore);
       FakeStack = createPHI(IRB, UseAfterReturnIsEnabled, FakeStackValue, Term,
                             ConstantInt::get(IntptrTy, 0));
     } else {
       // assert(ASan.UseAfterReturn == AsanDetectStackUseAfterReturnMode:Always)
       // void *FakeStack = __asan_stack_malloc_N(LocalStackSize);
       // void *LocalStackBase = (FakeStack) ? FakeStack :
       //                        alloca(LocalStackSize);
       StackMallocIdx = StackMallocSizeClass(LocalStackSize);
-      FakeStack = IRB.CreateCall(AsanStackMallocFunc[StackMallocIdx],
+      FakeStack =
+          ASan.createRuntimeCall(IRB, AsanStackMallocFunc[StackMallocIdx],
                                  ConstantInt::get(IntptrTy, LocalStackSize));
     }
     Value *NoFakeStack =
         IRB.CreateICmpEQ(FakeStack, Constant::getNullValue(IntptrTy));
     Instruction *Term =
         SplitBlockAndInsertIfThen(NoFakeStack, InsBefore, false);
     IRBuilder<> IRBIf(Term);
     Value *AllocaValue =
if (DoStackMalloc) {
             ConstantInt::get(IntptrTy, ClassSize - ASan.LongSize / 8));
         Value *SavedFlagPtr = IRBPoison.CreateLoad(
             IntptrTy, IRBPoison.CreateIntToPtr(SavedFlagPtrPtr, IntptrPtrTy));
         IRBPoison.CreateStore(
             Constant::getNullValue(IRBPoison.getInt8Ty()),
             IRBPoison.CreateIntToPtr(SavedFlagPtr, IRBPoison.getInt8PtrTy()));
       } else {
         // For larger frames call __asan_stack_free_*.
-        IRBPoison.CreateCall(
-            AsanStackFreeFunc[StackMallocIdx],
+        ASan.createRuntimeCall(
+            IRBPoison, AsanStackFreeFunc[StackMallocIdx],
             {FakeStack, ConstantInt::get(IntptrTy, LocalStackSize)});
       }
 
       IRBuilder<> IRBElse(ElseTerm);
       copyToShadow(ShadowAfterScope, ShadowClean, IRBElse, ShadowBase);
     } else {
       copyToShadow(ShadowAfterScope, ShadowClean, IRBRet, ShadowBase);
     }
   }
 
   // We are done. Remove the old unused alloca instructions.
   for (auto *AI : AllocaVec)
     AI->eraseFromParent();
 }
 
 void FunctionStackPoisoner::poisonAlloca(Value *V, uint64_t Size,
                                          IRBuilder<> &IRB, bool DoPoison) {
   // For now just insert the call to ASan runtime.
   Value *AddrArg = IRB.CreatePointerCast(V, IntptrTy);
   Value *SizeArg = ConstantInt::get(IntptrTy, Size);
-  IRB.CreateCall(
-      DoPoison ? AsanPoisonStackMemoryFunc : AsanUnpoisonStackMemoryFunc,
+  ASan.createRuntimeCall(
+      IRB, DoPoison ? AsanPoisonStackMemoryFunc : AsanUnpoisonStackMemoryFunc,
       {AddrArg, SizeArg});
 }
 
 // Handling llvm.lifetime intrinsics for a given %alloca:
 // (1) collect all llvm.lifetime.xxx(%size, %value) describing the alloca.
 // (2) if %size is constant, poison memory for llvm.lifetime.end (to detect
 //     invalid accesses) and unpoison it for llvm.lifetime.start (the memory
 //     could be poisoned by previous llvm.lifetime.end instruction, as the
void FunctionStackPoisoner::handleDynamicAllocaCall(AllocaInst *AI) {
   NewAlloca->setAlignment(Alignment);
 
   // NewAddress = Address + Alignment
   Value *NewAddress =
       IRB.CreateAdd(IRB.CreatePtrToInt(NewAlloca, IntptrTy),
                     ConstantInt::get(IntptrTy, Alignment.value()));
 
   // Insert __asan_alloca_poison call for new created alloca.
-  IRB.CreateCall(AsanAllocaPoisonFunc, {NewAddress, OldSize});
+  ASan.createRuntimeCall(IRB, AsanAllocaPoisonFunc, {NewAddress, OldSize});
 
   // Store the last alloca's address to DynamicAllocaLayout. We'll need this
   // for unpoisoning stuff.
   IRB.CreateStore(IRB.CreatePtrToInt(NewAlloca, IntptrTy), DynamicAllocaLayout);
 
   Value *NewAddressPtr = IRB.CreateIntToPtr(NewAddress, AI->getType());
 
   // Replace all uses of AddessReturnedByAlloca with NewAddressPtr.

llvm/test/Instrumentation/AddressSanitizer/asan-funclet.ll

+; Test appropriate tagging of funclet for function calls generated by asan.
+; RUN: opt -S -passes=asan -asan-max-inline-poisoning-size=0 \
+; RUN:   -asan-detect-invalid-pointer-cmp -asan-detect-invalid-pointer-sub -asan-use-after-scope < %s | FileCheck %s --check-prefixes=CHECK,CHECK-INLINE
+; RUN: opt -S -passes=asan -asan-max-inline-poisoning-size=0 -asan-instrumentation-with-call-threshold=0  \
+; RUN:   -asan-detect-invalid-pointer-cmp -asan-detect-invalid-pointer-sub -asan-use-after-scope < %s | FileCheck %s --check-prefixes=CHECK,CHECK-OUTLINE
+
+; REQUIRES: x86-registered-target
+
+target triple = "x86_64-pc-windows-msvc"
+
+declare void @DeInit(ptr)
+declare void @MayThrowFunc()
+declare void @NoReturn() noreturn
+
+declare void @llvm.memmove.p0.p0.i64(ptr nocapture, ptr nocapture readonly, i64, i1)
+declare void @llvm.memcpy.p0.p0.i64(ptr noalias nocapture writeonly, ptr noalias nocapture readonly, i64, i1)
+declare void @llvm.memset.p0.i64(ptr nocapture writeonly, i8, i64, i1)
+declare void @llvm.lifetime.start.p0(i64, ptr nocapture) nounwind
+declare void @llvm.lifetime.end.p0(i64, ptr nocapture) nounwind
+
+declare i32 @__CxxFrameHandler3(...)
+declare i32 @dummyPersonality(...)
+
+define void @FuncletPersonality(ptr %ptrParam) sanitize_address personality ptr @__CxxFrameHandler3 {
+; CHECK-LABEL: @FuncletPersonality
+; CHECK: ehcleanup:
+; CHECK: [[CleanupPad1:%[^ ]+]] = cleanuppad within none []
+; CHECK-INLINE: call void @__asan_unpoison_stack_memory{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+; CHECK-INLINE: call void @__asan_report_store1{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+; CHECK-INLINE: call void @__asan_poison_stack_memory{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+; CHECK: call void @DeInit({{.*}}) [ "funclet"(token [[CleanupPad1]]) ]
+; CHECK: call ptr @__asan_memset{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+; CHECK: call ptr @__asan_memcpy{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+; CHECK: call ptr @__asan_memmove{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+; CHECK: call void @__sanitizer_ptr_cmp{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+; CHECK: call void @__sanitizer_ptr_sub{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+
+; CHECK-OUTLINE: call void @__asan_storeN{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+
+; CHECK: noreturncall:
+; CHECK: call void @__asan_handle_no_return{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+; CHECK: ehexit:
+; CHECK: call void @__asan_allocas_unpoison{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+; CHECK: call void @__asan_stack_free_{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+; CHECK: call void @__asan_set_shadow_{{.*}} [ "funclet"(token [[CleanupPad1]]) ]
+entry:
+  ; Large enough local alloca to have asan generate a __asan_stack_free_#() call
+  %largeObj = alloca [2048 x i32], align 16
+  %tmpInt1 = alloca i32, align 4
+  %tmpInt2 = alloca i32, align 4
+  %tmpInt3 = alloca i32, align 4
+
+  ; Creating %lifetimeInt and %lifetimeArr, and managing their lifetimes
+  ; to make asan generate stack poisoning calls
+  %lifetimeInt = alloca i32, align 4
+  call void @llvm.lifetime.start.p0(i64 4, ptr %lifetimeInt)
+  store volatile i8 0, ptr %lifetimeInt
+  call void @llvm.lifetime.end.p0(i64 4, ptr %lifetimeInt)
+  %lifetimeArr = alloca i32, align 4
+
+  ; Dynamic alloca to generate a @__asan_allocas_unpoison call in ehcleanup
+  %tmpVolatilei64 = alloca i64, align 8
+  store volatile i64 0, ptr %tmpVolatilei64, align 8
+  %tmpCopyi64 = load i64, ptr %tmpVolatilei64, align 8
+  %tmpVolatilei8 = alloca i8, i64 %tmpCopyi64, align 32
+  store volatile i8 0, ptr %tmpVolatilei8
+
+  invoke void @MayThrowFunc()
+          to label %invoke.cont unwind label %ehcleanup
+invoke.cont:                                      ; preds = %entry
+  call void @DeInit(ptr %largeObj)
+  ret void
+
+ehcleanup:                                        ; preds = %entry
+  %0 = cleanuppad within none []
+
+  ; Make asan add a call to __asan_unpoison_stack_memory
+  call void @llvm.lifetime.start.p0(i64 4, ptr %lifetimeArr)
+  ; Make asan add a call to __asan_report_store1
+  store volatile i8 0, ptr %lifetimeArr
+  ; Make asan add a call to __asan_poison_stack_memory
+  call void @llvm.lifetime.end.p0(i64 4, ptr %lifetimeArr)
+
+  call void @DeInit(ptr %largeObj) [ "funclet"(token %0) ]
+  call void @llvm.memset.p0.i64(ptr align 4 %tmpInt1, i8 0, i64 4, i1 false)
+  call void @llvm.memcpy.p0.p0.i64(ptr align 4 %tmpInt2, ptr align 4 %tmpInt1, i64 4, i1 false)
+  call void @llvm.memmove.p0.p0.i64(ptr align 4 %tmpInt3, ptr align 4 %tmpInt1, i64 4, i1 false)
+  %cmpAddr = icmp ule ptr %tmpInt1, %tmpInt2
+  %addr1 = ptrtoint ptr %tmpInt1 to i64
+  %addr2 = ptrtoint ptr %tmpInt2 to i64
+  %subAddr = sub i64 %addr1, %addr2
+
+  store i64 0, ptr %ptrParam, align 1
+
+  %cmp = icmp ne i64 %subAddr, 0
+  br i1 %cmp, label %ehexit, label %noreturncall
+
+noreturncall:
+  call void @NoReturn(ptr null, ptr null) noreturn [ "funclet"(token %0) ]
+  unreachable
+
+ehexit:
+  cleanupret from %0 unwind to caller
+
+; Ensure unreachable basic block doesn't make the compiler assert, as it's a special case for coloring computation.
+nopredecessor:
+  call void @llvm.memset.p0.i64(ptr align 4 %tmpInt1, i8 0, i64 4, i1 false)
+  unreachable
+}
+
+; Non-Windows personality, ensure no funclet gets attached to asan runtime call.
+define void @OtherPersonality(ptr %ptrParam) sanitize_address personality ptr @dummyPersonality {
+; CHECK-LABEL: @OtherPersonality
+; CHECK: ehcleanup:
+; CHECK: call ptr @__asan_memset
+; CHECK-NOT: funclet
+entry:
+  %tmpInt = alloca i32, align 4
+  invoke void @MayThrowFunc()
+          to label %invoke.cont unwind label %ehcleanup
+invoke.cont:                                      ; preds = %entry
+  ret void
+
+ehcleanup:                                        ; preds = %entry
+  %0 = cleanuppad within none []
+  call void @llvm.memset.p0.i64(ptr align 4 %tmpInt, i8 0, i64 4, i1 false)
+  cleanupret from %0 unwind to caller
+}

llvm/test/Instrumentation/AddressSanitizer/localescape.ll

 ; RUN: opt < %s -passes=asan -asan-use-after-return=never -asan-stack-dynamic-alloca -S | FileCheck %s
 ; RUN: opt < %s -passes=asan -asan-use-after-return=runtime -asan-stack-dynamic-alloca -S | FileCheck %s
 ; RUN: opt < %s -passes=asan -asan-use-after-return=always -asan-stack-dynamic-alloca -S | FileCheck %s
 ; RUN: opt < %s -passes=asan -asan-use-after-return=never -asan-stack-dynamic-alloca=0 -S | FileCheck %s
 ; RUN: opt < %s -passes=asan -asan-use-after-return=runtime -asan-stack-dynamic-alloca=0 -S | FileCheck %s
 ; RUN: opt < %s -passes=asan -asan-use-after-return=always -asan-stack-dynamic-alloca=0 -S | FileCheck %s
 
 target datalayout = "e-m:x-p:32:32-i64:64-f80:32-n8:16:32-a:0:32-S32"
 target triple = "i686-pc-windows-msvc18.0.0"
 
 declare i32 @llvm.eh.typeid.for(ptr) #2
 declare ptr @llvm.frameaddress(i32)
 declare ptr @llvm.eh.recoverfp(ptr, ptr)
 declare ptr @llvm.localrecover(ptr, ptr, i32)
 declare void @llvm.localescape(...) #1
 
-declare i32 @_except_handler3(...)
+declare i32 @__gcc_personality_v0(...)
 declare void @may_throw(ptr %r)
 
-define i32 @main() sanitize_address personality ptr @_except_handler3 {
+define i32 @main() sanitize_address personality ptr @__gcc_personality_v0 {
 entry:
   %r = alloca i32, align 4
   %__exception_code = alloca i32, align 4
   call void (...) @llvm.localescape(ptr nonnull %__exception_code)
   store i32 0, ptr %r, align 4
   invoke void @may_throw(ptr nonnull %r) #4
           to label %__try.cont unwind label %lpad