[ASan] Skip any instruction inserted by another instrumentation.

[Enna1]

Summary

Currently, we only check !nosanitize metadata for instruction passed to function getInterestingMemoryOperands() or instruction which is a cannot return callable instruction.
This patch add this check to any instruction.

E.g. ASan shouldn't instrument the instruction inserted by UBSan/pointer-overflow.

[vitalybuka]

We need a test here as well

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

2743

Shouldn't we skip any instruction with nosanitize?

[Enna1]

address review comments

[Enna1]

Update patch

[Enna1]

I think we can remove D126270 as parent revision, only add D126294 as parent revision, and land this patch regardless of D126270 ?

[Enna1]

Gentle ping

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

 static bool isPointerOperand(Value *V) {
   return V->getType()->isPointerTy() || isa<PtrToIntInst>(V);
 }
 
 // This is a rough heuristic; it may cause both false positives and
 // false negatives. The proper implementation requires cooperation with
 // the frontend.
 static bool isInterestingPointerComparison(Instruction *I) {
+  if (I->hasMetadata("nosanitize"))
+    return false;
   if (ICmpInst *Cmp = dyn_cast<ICmpInst>(I)) {
     if (!Cmp->isRelational())
       return false;
   } else {
     return false;
   }
   return isPointerOperand(I->getOperand(0)) &&
          isPointerOperand(I->getOperand(1));
 }
 
 // This is a rough heuristic; it may cause both false positives and
 // false negatives. The proper implementation requires cooperation with
 // the frontend.
 static bool isInterestingPointerSubtraction(Instruction *I) {
+  if (I->hasMetadata("nosanitize"))
+    return false;
   if (BinaryOperator *BO = dyn_cast<BinaryOperator>(I)) {
     if (BO->getOpcode() != Instruction::Sub)
       return false;
   } else {
     return false;
   }
   return isPointerOperand(I->getOperand(0)) &&
          isPointerOperand(I->getOperand(1));
bool AddressSanitizer::instrumentFunction(Function &F,
   SmallVector<Instruction *, 16> PointerComparisonsOrSubtracts;
 
   // Fill the set of memory operations to instrument.
   for (auto &BB : F) {
     AllBlocks.push_back(&BB);
     TempsToInstrument.clear();
     int NumInsnsPerBB = 0;
     for (auto &Inst : BB) {
       if (LooksLikeCodeInBug11395(&Inst)) return false;
       SmallVector<InterestingMemoryOperand, 1> InterestingOperands;
       getInterestingMemoryOperands(&Inst, InterestingOperands);
 
       if (!InterestingOperands.empty()) {
         for (auto &Operand : InterestingOperands) {
           if (ClOpt && ClOptSameTemp) {
             Value *Ptr = Operand.getPtr();
             // If we have a mask, skip instrumentation if we've already