[asan] Reports suppressions for ASan recovery mode (LLVM core part).

[m.ostapenko]

Summary

Hi!

When we run sanitized application in ASan recovery mode, sometimes we have lots of identical reports for the same error (e.g. error in hot loop). It would be nice to filter out such reports.
We can use the same approach as for UBSan with disabling source locations to achieve this goal.

This is the LLVM part of suppression functionality. Here, we just create SourceLocation descriptor for each memory access and provide a pointer to it for asan_report*_noabort functions.
There aren't any changes in default halt_on_error mode.

[ygribov]

lib/Transforms/Instrumentation/AddressSanitizer.cpp

1017

Is it possible move call out of if-else and set argument ArrayRefs there instead? Same for other places below.

1469

Perhaps used nested conditional operator?

[kcc]

Can't we just rely on __builtin_return_address to deduplicate reports?

The was this is done in ubsan is not necassary the best way for asan.
ubsan was initially designed to work with no or very lightweight run-time.

[ygribov]

Can't we just rely on __builtin_return_address to deduplicate reports?

You mean poison shadow of return address with some meaningful value? This would work although I'm scared of shared libraries (e.g. if they are unmapped, etc.).

[kcc]

You mean poison shadow of return address with some meaningful value? This would work although I'm scared of shared libraries (e.g. if they are unmapped, etc.).

No. You can just remember the return address in an array, and when reporting another bug consult that array.

[m.ostapenko]

In D15079#298845, @kcc wrote:

You mean poison shadow of return address with some meaningful value? This would work although I'm scared of shared libraries (e.g. if they are unmapped, etc.).

No. You can just remember the return address in an array, and when reporting another bug consult that array.

E.g. remember, say, last ten return addresses into array and just lookup there for given caller PC? This sounds reasonable for me.

[kcc]

In D15079#298852, @m.ostepenko wrote:

In D15079#298845, @kcc wrote:

You mean poison shadow of return address with some meaningful value? This would work although I'm scared of shared libraries (e.g. if they are unmapped, etc.).

No. You can just remember the return address in an array, and when reporting another bug consult that array.

E.g. remember, say, last ten return addresses into array and just lookup there for given caller PC? This sounds reasonable for me.

I would say remember all, and bail out if there are more than 100 different PCs (in which case running further is insane).

[ygribov]

I would say remember all, and bail out if there are more than 100 different PCs (in which case running further is insane).

Hm but linear scan through a 100-element array would be slow.

[m.ostapenko]

In D15079#299155, @ygribov wrote:

I would say remember all, and bail out if there are more than 100 different PCs (in which case running further is insane).

Hm but linear scan through a 100-element array would be slow.

I think we can use less threshold value.
Another issue is races on shared array. Should we use mutex or perhaps it's OK to sacrifice a bit of accuracy here?

[ygribov]

I hope atomic counter will be enough.

[m.ostapenko]

Ok, with discussed approach we don't need changes in LLVM part, only in compiler-rt. So, abandoning this revision, let's move further discussion to compiler-rt part (http://reviews.llvm.org/D15080).

lib/Transforms/Instrumentation/AddressSanitizer.cpp

struct AddressSanitizer : public FunctionPass {
   void instrumentUnusualSizeOrAlignment(Instruction *I, Value *Addr,
                                         uint32_t TypeSize, bool IsWrite,
                                         Value *SizeArgument, bool UseCalls,
                                         uint32_t Exp);
   Value *createSlowPathCmp(IRBuilder<> &IRB, Value *AddrLong,
                            Value *ShadowValue, uint32_t TypeSize);
   Instruction *generateCrashCode(Instruction *InsertBefore, Value *Addr,
                                  bool IsWrite, size_t AccessSizeIndex,
-                                 Value *SizeArgument, uint32_t Exp);
+                                 Value *SizeArgument, uint32_t Exp,
+                                 Value *SourceLoc);
   void instrumentMemIntrinsic(MemIntrinsic *MI);
   Value *memToShadow(Value *Shadow, IRBuilder<> &IRB);
   bool runOnFunction(Function &F) override;
   bool maybeInsertAsanInitAtFunctionEntry(Function &F);
   void markEscapedLocalAllocas(Function &F);
   bool doInitialization(Module &M) override;
   static char ID;  // Pass identification, replacement for typeid
 
GlobalVariable *GV =
       new GlobalVariable(M, StrConst->getType(), true,
                          GlobalValue::PrivateLinkage, StrConst, kAsanGenPrefix);
   if (AllowMerging) GV->setUnnamedAddr(true);
   GV->setAlignment(1);  // Strings may not be merged w/o setting align 1.
   return GV;
 }
 
 /// \brief Create a global describing a source location.
-static GlobalVariable *createPrivateGlobalForSourceLoc(Module &M,
-                                                       LocationMetadata MD) {
+static GlobalVariable *createGlobalForSourceLoc(Module &M,
+                                                LocationMetadata MD,
+                                                bool isPrivate) {
   Constant *LocData[] = {
       createPrivateGlobalForString(M, MD.Filename, true),
       ConstantInt::get(Type::getInt32Ty(M.getContext()), MD.LineNo),
       ConstantInt::get(Type::getInt32Ty(M.getContext()), MD.ColumnNo),
   };
   auto LocStruct = ConstantStruct::getAnon(LocData);
-  auto GV = new GlobalVariable(M, LocStruct->getType(), true,
+  auto GV = new GlobalVariable(M, LocStruct->getType(), isPrivate,
                                GlobalValue::PrivateLinkage, LocStruct,
                                kAsanGenPrefix);
   GV->setUnnamedAddr(true);
   return GV;
 }
 
 static bool GlobalWasGeneratedByAsan(GlobalVariable *G) {
   return G->getName().find(kAsanGenPrefix) == 0 ||
void AddressSanitizer::instrumentMop(ObjectSizeOffsetVisitor &ObjSizeVis,
   instrumentUnusualSizeOrAlignment(I, Addr, TypeSize, IsWrite, nullptr,
                                    UseCalls, Exp);
 }
 
 Instruction *AddressSanitizer::generateCrashCode(Instruction *InsertBefore,
                                                  Value *Addr, bool IsWrite,
                                                  size_t AccessSizeIndex,
                                                  Value *SizeArgument,
-                                                 uint32_t Exp) {
+                                                 uint32_t Exp,
+                                                 Value *SourceLoc) {
   IRBuilder<> IRB(InsertBefore);
   Value *ExpVal = Exp == 0 ? nullptr : ConstantInt::get(IRB.getInt32Ty(), Exp);
   CallInst *Call = nullptr;
   if (SizeArgument) {
     if (Exp == 0)
+      if (Recover)
+        Call = IRB.CreateCall(AsanErrorCallbackSized[IsWrite][0],
+                              {Addr, SizeArgument, SourceLoc});
+      else
         Call = IRB.CreateCall(AsanErrorCallbackSized[IsWrite][0],
                               {Addr, SizeArgument});
     else
       Call = IRB.CreateCall(AsanErrorCallbackSized[IsWrite][1],
                             {Addr, SizeArgument, ExpVal});
   } else {
     if (Exp == 0)
-      Call =
-          IRB.CreateCall(AsanErrorCallback[IsWrite][0][AccessSizeIndex], Addr);
+      if (Recover)
+        Call = IRB.CreateCall(AsanErrorCallback[IsWrite][0][AccessSizeIndex],
+                              {Addr, SourceLoc});
+      else
+        Call = IRB.CreateCall(AsanErrorCallback[IsWrite][0][AccessSizeIndex],
+                              {Addr});
     else
       Call = IRB.CreateCall(AsanErrorCallback[IsWrite][1][AccessSizeIndex],
                             {Addr, ExpVal});
   }
 
   // We don't do Call->setDoesNotReturn() because the BB already has
   // UnreachableInst at the end.
   // This EmptyAsm is required to avoid callback merge.
void AddressSanitizer::instrumentAddress(Instruction *OrigIns,
                                          Instruction *InsertBefore, Value *Addr,
                                          uint32_t TypeSize, bool IsWrite,
                                          Value *SizeArgument, bool UseCalls,
                                          uint32_t Exp) {
   IRBuilder<> IRB(InsertBefore);
   Value *AddrLong = IRB.CreatePointerCast(Addr, IntptrTy);
   size_t AccessSizeIndex = TypeSizeToSizeIndex(TypeSize);
 
+  Value *SourceLoc = nullptr;
+  if (Recover) {
+    Function *F = OrigIns->getParent()->getParent();
+    Module *M = F->getParent();
+    GlobalVariable *SourceLocDescr =
+        createGlobalForSourceLoc(*M, LocationMetadata(), false);
+    SourceLoc = IRB.CreatePointerCast(SourceLocDescr, IntptrTy);
+  }
+
   if (UseCalls) {
     if (Exp == 0)
+      if (Recover)
+        IRB.CreateCall(AsanMemoryAccessCallback[IsWrite][0][AccessSizeIndex],
+                       {AddrLong, SourceLoc});
+      else
         IRB.CreateCall(AsanMemoryAccessCallback[IsWrite][0][AccessSizeIndex],
                        AddrLong);
     else
       IRB.CreateCall(AsanMemoryAccessCallback[IsWrite][1][AccessSizeIndex],
                      {AddrLong, ConstantInt::get(IRB.getInt32Ty(), Exp)});
     return;
   }
 
   Type *ShadowTy =
       IntegerType::get(*C, std::max(8U, TypeSize >> Mapping.Scale));
if (Recover) {
       CrashTerm = new UnreachableInst(*C, CrashBlock);
       BranchInst *NewTerm = BranchInst::Create(CrashBlock, NextBB, Cmp2);
       ReplaceInstWithInst(CheckTerm, NewTerm);
     }
   } else {
     CrashTerm = SplitBlockAndInsertIfThen(Cmp, InsertBefore, !Recover);
   }
 
-  Instruction *Crash = generateCrashCode(CrashTerm, AddrLong, IsWrite,
-                                         AccessSizeIndex, SizeArgument, Exp);
+  Instruction *Crash =
+      generateCrashCode(CrashTerm, AddrLong, IsWrite, AccessSizeIndex,
+                        SizeArgument, Exp, SourceLoc);
   Crash->setDebugLoc(OrigIns->getDebugLoc());
 }
 
 // Instrument unusual size or unusual alignment.
 // We can not do it with a single check, so we do 1-byte check for the first
 // and the last bytes. We call __asan_report_*_n(addr, real_size) to be able
 // to report the actual access size.
 void AddressSanitizer::instrumentUnusualSizeOrAlignment(
for (size_t i = 0; i < n; i++) {
 
     G->replaceAllUsesWith(
         ConstantExpr::getGetElementPtr(NewTy, NewGlobal, Indices2, true));
     NewGlobal->takeName(G);
     G->eraseFromParent();
 
     Constant *SourceLoc;
     if (!MD.SourceLoc.empty()) {
-      auto SourceLocGlobal = createPrivateGlobalForSourceLoc(M, MD.SourceLoc);
+      auto SourceLocGlobal = createGlobalForSourceLoc(M, MD.SourceLoc, true);
       SourceLoc = ConstantExpr::getPointerCast(SourceLocGlobal, IntptrTy);
     } else {
       SourceLoc = ConstantInt::get(IntptrTy, 0);
     }
 
     Initializers[i] = ConstantStruct::get(
         GlobalStructTy, ConstantExpr::getPointerCast(NewGlobal, IntptrTy),
         ConstantInt::get(IntptrTy, SizeInBytes),
if (ClGlobals && !CompileKernel) {
     Changed |= InstrumentGlobals(IRB, M);
   }
 
   return Changed;
 }
 
 void AddressSanitizer::initializeCallbacks(Module &M) {
   IRBuilder<> IRB(*C);
+  assert (!(Exp && Recover) && "ASan doesn't support experiments in recover "
+                               "mode!");
   // Create __asan_report* callbacks.
   // IsWrite, TypeSize and Exp are encoded in the function name.
   for (int Exp = 0; Exp < 2; Exp++) {
     for (size_t AccessIsWrite = 0; AccessIsWrite <= 1; AccessIsWrite++) {
       const std::string TypeStr = AccessIsWrite ? "store" : "load";
       const std::string ExpStr = Exp ? "exp_" : "";
       const std::string SuffixStr = CompileKernel ? "N" : "_n";
       const std::string EndingStr = Recover ? "_noabort" : "";
-      Type *ExpType = Exp ? Type::getInt32Ty(*C) : nullptr;
+      Type *OptType = nullptr;
+      if (Exp)
+        OptType = Type::getInt32Ty(*C);
+      else if (Recover)
+        OptType = IntptrTy;
       AsanErrorCallbackSized[AccessIsWrite][Exp] =
           checkSanitizerInterfaceFunction(M.getOrInsertFunction(
               kAsanReportErrorTemplate + ExpStr + TypeStr + SuffixStr + EndingStr,
-              IRB.getVoidTy(), IntptrTy, IntptrTy, ExpType, nullptr));
+              IRB.getVoidTy(), IntptrTy, IntptrTy, OptType, nullptr));
       AsanMemoryAccessCallbackSized[AccessIsWrite][Exp] =
           checkSanitizerInterfaceFunction(M.getOrInsertFunction(
               ClMemoryAccessCallbackPrefix + ExpStr + TypeStr + "N" + EndingStr,
-              IRB.getVoidTy(), IntptrTy, IntptrTy, ExpType, nullptr));
+              IRB.getVoidTy(), IntptrTy, IntptrTy, OptType, nullptr));
       for (size_t AccessSizeIndex = 0; AccessSizeIndex < kNumberOfAccessSizes;
            AccessSizeIndex++) {
         const std::string Suffix = TypeStr + itostr(1 << AccessSizeIndex);
         AsanErrorCallback[AccessIsWrite][Exp][AccessSizeIndex] =
             checkSanitizerInterfaceFunction(M.getOrInsertFunction(
                 kAsanReportErrorTemplate + ExpStr + Suffix + EndingStr,
-                IRB.getVoidTy(), IntptrTy, ExpType, nullptr));
+                IRB.getVoidTy(), IntptrTy, OptType, nullptr));
         AsanMemoryAccessCallback[AccessIsWrite][Exp][AccessSizeIndex] =
             checkSanitizerInterfaceFunction(M.getOrInsertFunction(
                 ClMemoryAccessCallbackPrefix + ExpStr + Suffix + EndingStr,
-                IRB.getVoidTy(), IntptrTy, ExpType, nullptr));
+                IRB.getVoidTy(), IntptrTy, OptType, nullptr));
       }
     }
   }
 
   const std::string MemIntrinCallbackPrefix =
       CompileKernel ? std::string("") : ClMemoryAccessCallbackPrefix;
   AsanMemmove = checkSanitizerInterfaceFunction(M.getOrInsertFunction(
       MemIntrinCallbackPrefix + "memmove", IRB.getInt8PtrTy(),