[asan] Implemented flag to emit intrinsics to optimize ASan callbacks.

[kstoimenov]

Pulled constants like shadow base, etc out of the intrinsic.

[vitalybuka]

llvm/include/llvm/Transforms/Instrumentation/AddressSanitizer.h

151 ↗

(On Diff #367500)

Could please please replace Module with targetTriple and pointerSizeInBits.
We don't need entire module for that even if current callers have it.

llvm/include/llvm/Transforms/Instrumentation/AddressSanitizerCommon.h

90 ↗

(On Diff #367500)

I don't see implementation of this one.

And I expected getAddressSanitizerParams here, not in AddressSanitizer.h. Is there a reson to declare it there?

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

574

It would be nice to introduce this function as a separate NFC patch.

[vitalybuka]

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

579

Unfortunately we need Kernel as argument, similar to RecoverShift in HWAsan.
Look at CL flags as internal per process constants. So if value is controlled by them then it's OK to assume the same value.
Kernel value however can be set by fronted e.g. from clang and does not match ClEnableKasan.

I propose to have own set of intrinsics for kernel/non-kernel to avoid fancy bit packing or unreadable 1/0 arguments.
Later we can cleunup hwasan as well.

[kstoimenov]

Update before expanding intrinsics.

[kstoimenov]

Removed unused encodeMemToShadowInfo.

[vitalybuka]

clang/test/CodeGen/asan-use-callbacks.cpp

9 ↗

(On Diff #367585)

for -mllvm flag we need test under llvm/test not clang.

llvm test needs to be more meaningful, e.g. check precice value passed into intrinsic
and have 3 versions:
... -mllvm -asan-optimize-callbacks=0
... -mllvm -asan-optimize-callbacks=1
... default

[vitalybuka]

clang/test/CodeGen/asan-use-callbacks.cpp

9 ↗

(On Diff #367585)

to clarify
Goal of clang/test/CodeGen/asan-use-callbacks.cpp to test -fsanitize-address-outline-instrumentation which is frontend (clang) flag.
-mllvm are not clang flags, even clang knows how to forward them to llvm.

I believe we will not add corresponding frontend clang flag for asan-optimize-callbacks. We will make it default ON after some testing and benchmarking.

[kstoimenov]

Removed test code which should be added under a different directory.

[vitalybuka]

Flag is not used anywhere?

[kstoimenov]

Moved files from the parent patch which didn't belong there.

[kstoimenov]

This time AddressSanitizer.cpp should be updated for real.

[vitalybuka]

The test is still missing?

[kstoimenov]

Attempt to split AddressSanitizer.cpp.

[kstoimenov]

Tests are still WIP. This is not ready for review yet. I will ping you when it it.

[vitalybuka]

In D108377#2962445, @kstoimenov wrote:

Tests are still WIP. This is not ready for review yet. I will ping you when it it.

There is also "Add Acction..."-> "Plan changes", which will remove it from reviewers dashboards.

[kstoimenov]

Thanks!

[kstoimenov]

Updating after pushing D107850.

[kstoimenov]

Upload after submitting the main patch.

[kstoimenov]

Added a test.

[kstoimenov]

Fixed the test.

[vitalybuka]

llvm/test/Instrumentation/AddressSanitizer/asan-optimize-callbacks.ll

7

Can you please include negative cases as well?

; RUN: opt < %s -asan -enable-new-pm=0 -asan-instrumentation-with-call-threshold=0 -asan-optimize-callbacks=0 -S | FileCheck %s --implicit-check-not llvm.asan.check.memaccess
; RUN: opt < %s -asan -enable-new-pm=0 -asan-instrumentation-with-call-threshold=0 -asan-optimize-callbacks=0 --asan-kernel -S | FileCheck %s --implicit-check-not llvm.asan.check.memaccess

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

ClMappingOffset("asan-mapping-offset",
                     cl::Hidden, cl::init(0));
 
 // Optimization flags. Not user visible, used mostly for testing
 // and benchmarking the tool.
 
 static cl::opt<bool> ClOpt("asan-opt", cl::desc("Optimize instrumentation"),
                            cl::Hidden, cl::init(true));
 
+static cl::opt<bool> ClOptimizeCallbacks("asan-optimize-callbacks",
+                                         cl::desc("Optimize callbacks"),
+                                         cl::Hidden, cl::init(false));
+
 static cl::opt<bool> ClOptSameTemp(
     "asan-opt-same-temp", cl::desc("Instrument the same temp just once"),
     cl::Hidden, cl::init(true));
 
 static cl::opt<bool> ClOptGlobals("asan-opt-globals",
                                   cl::desc("Don't instrument scalar globals"),
                                   cl::Hidden, cl::init(true));
 
Mapping.OrShadowOffset = !IsAArch64 && !IsPPC64 && !IsSystemZ && !IsPS4CPU &&
                            Mapping.Offset != kDynamicShadowSentinel;
   bool IsAndroidWithIfuncSupport =
       IsAndroid && !TargetTriple.isAndroidVersionLT(21);
   Mapping.InGlobal = ClWithIfunc && IsAndroidWithIfuncSupport && IsArmOrThumb;
 
   return Mapping;
 }
 
 namespace llvm {
 void getAddressSanitizerParams(const Triple &TargetTriple, int LongSize,
                                bool IsKasan, uint64_t *ShadowBase,
                                int *MappingScale, bool *OrShadowOffset) {
   auto Mapping = getShadowMapping(TargetTriple, LongSize, IsKasan);
   *ShadowBase = Mapping.Offset;
   *MappingScale = Mapping.Scale;
   *OrShadowOffset = Mapping.OrShadowOffset;
 }
 
 ASanAccessInfo::ASanAccessInfo(int32_t Packed)
     : Packed(Packed),
       AccessSizeIndex((Packed >> kAccessSizeIndexShift) & kAccessSizeIndexMask),
       IsWrite((Packed >> kIsWriteShift) & kIsWriteMask),
AddressSanitizer(Module &M, const GlobalsMetadata *GlobalsMD,
         Recover(ClRecover.getNumOccurrences() > 0 ? ClRecover : Recover),
         UseAfterScope(UseAfterScope || ClUseAfterScope),
         UseAfterReturn(ClUseAfterReturn.getNumOccurrences() ? ClUseAfterReturn
                                                             : UseAfterReturn),
         GlobalsMD(*GlobalsMD) {
     C = &(M.getContext());
     LongSize = M.getDataLayout().getPointerSizeInBits();
     IntptrTy = Type::getIntNTy(*C, LongSize);
+    Int8PtrTy = Type::getInt8PtrTy(*C);
+    Int32Ty = Type::getInt32Ty(*C);
     TargetTriple = Triple(M.getTargetTriple());
 
     Mapping = getShadowMapping(TargetTriple, LongSize, this->CompileKernel);
 
     assert(this->UseAfterReturn != AsanDetectStackUseAfterReturnMode::Invalid);
   }
 
   uint64_t getAllocaSizeInBytes(const AllocaInst &AI) const {
private:
   LLVMContext *C;
   Triple TargetTriple;
   int LongSize;
   bool CompileKernel;
   bool Recover;
   bool UseAfterScope;
   AsanDetectStackUseAfterReturnMode UseAfterReturn;
   Type *IntptrTy;
+  Type *Int8PtrTy;
+  Type *Int32Ty;
   ShadowMapping Mapping;
   FunctionCallee AsanHandleNoReturnFunc;
   FunctionCallee AsanPtrCmpFunction, AsanPtrSubFunction;
   Constant *AsanShadowGlobal;
 
   // These arrays is indexed by AccessIsWrite, Experiment and log2(AccessSize).
   FunctionCallee AsanErrorCallback[2][2][kNumberOfAccessSizes];
   FunctionCallee AsanMemoryAccessCallback[2][2][kNumberOfAccessSizes];
void AddressSanitizer::instrumentAddress(Instruction *OrigIns,
   if (TargetTriple.isAMDGPU()) {
     InsertBefore = instrumentAMDGPUAddress(OrigIns, InsertBefore, Addr,
                                            TypeSize, IsWrite, SizeArgument);
     if (!InsertBefore)
       return;
   }
 
   IRBuilder<> IRB(InsertBefore);
-  Value *AddrLong = IRB.CreatePointerCast(Addr, IntptrTy);
   size_t AccessSizeIndex = TypeSizeToSizeIndex(TypeSize);
+  const ASanAccessInfo AccessInfo(IsWrite, CompileKernel, AccessSizeIndex);
+
+  if (UseCalls && ClOptimizeCallbacks) {
+    const ASanAccessInfo AccessInfo(IsWrite, CompileKernel, AccessSizeIndex);
+    Module *M = IRB.GetInsertBlock()->getParent()->getParent();
+    IRB.CreateCall(
+        Intrinsic::getDeclaration(M, Intrinsic::asan_check_memaccess),
+        {IRB.CreatePointerCast(Addr, Int8PtrTy),
+         ConstantInt::get(Int32Ty, AccessInfo.Packed)});
+    return;
+  }
 
+  Value *AddrLong = IRB.CreatePointerCast(Addr, IntptrTy);
   if (UseCalls) {
     if (Exp == 0)
       IRB.CreateCall(AsanMemoryAccessCallback[IsWrite][0][AccessSizeIndex],
                      AddrLong);
     else
       IRB.CreateCall(AsanMemoryAccessCallback[IsWrite][1][AccessSizeIndex],
                      {AddrLong, ConstantInt::get(IRB.getInt32Ty(), Exp)});
     return;

llvm/test/Instrumentation/AddressSanitizer/asan-optimize-callbacks.ll

+; RUN: opt < %s -asan -enable-new-pm=0 -asan-instrumentation-with-call-threshold=0 \
+; RUN:   -asan-optimize-callbacks -S | FileCheck %s --check-prefixes=LOAD,STORE
+; RUN: opt < %s -asan -enable-new-pm=0 -asan-instrumentation-with-call-threshold=0 \
+; RUN:   -asan-optimize-callbacks --asan-kernel -S | \
+; RUN:   FileCheck %s --check-prefixes=LOAD-KERNEL,STORE-KERNEL
+
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @load(i8* %p1, i16* %p2, i32* %p4, i64* %p8, i128* %p16)
+sanitize_address {
+  %n1 = load i8, i8* %p1, align 1
+  %n2 = load i16, i16* %p2, align 2
+  %n4 = load i32, i32* %p4, align 4
+  %n8 = load i64, i64* %p8, align 8
+  %n16 = load i128, i128* %p16, align 16
+; LOAD:      call void @llvm.asan.check.memaccess(i8* %p1, i32 0)
+; LOAD-NEXT: %n1 = load i8, i8* %p1, align 1
+; LOAD-NEXT: %1 = bitcast i16* %p2 to i8*
+; LOAD-NEXT: call void @llvm.asan.check.memaccess(i8* %1, i32 2)
+; LOAD-NEXT: %n2 = load i16, i16* %p2, align 2
+; LOAD-NEXT: %2 = bitcast i32* %p4 to i8*
+; LOAD-NEXT: call void @llvm.asan.check.memaccess(i8* %2, i32 4)
+; LOAD-NEXT: %n4 = load i32, i32* %p4, align 4
+; LOAD-NEXT: %3 = bitcast i64* %p8 to i8*
+; LOAD-NEXT: call void @llvm.asan.check.memaccess(i8* %3, i32 6)
+; LOAD-NEXT: %n8 = load i64, i64* %p8, align 8
+; LOAD-NEXT: %4 = bitcast i128* %p16 to i8*
+; LOAD-NEXT: call void @llvm.asan.check.memaccess(i8* %4, i32 8)
+; LOAD-NEXT: %n16 = load i128, i128* %p16, align 16
+
+; LOAD-KERNEL:      call void @llvm.asan.check.memaccess(i8* %p1, i32 1)
+; LOAD-KERNEL-NEXT: %n1 = load i8, i8* %p1, align 1
+; LOAD-KERNEL-NEXT: %1 = bitcast i16* %p2 to i8*
+; LOAD-KERNEL-NEXT: call void @llvm.asan.check.memaccess(i8* %1, i32 3)
+; LOAD-KERNEL-NEXT: %n2 = load i16, i16* %p2, align 2
+; LOAD-KERNEL-NEXT: %2 = bitcast i32* %p4 to i8*
+; LOAD-KERNEL-NEXT: call void @llvm.asan.check.memaccess(i8* %2, i32 5)
+; LOAD-KERNEL-NEXT: %n4 = load i32, i32* %p4, align 4
+; LOAD-KERNEL-NEXT: %3 = bitcast i64* %p8 to i8*
+; LOAD-KERNEL-NEXT: call void @llvm.asan.check.memaccess(i8* %3, i32 7)
+; LOAD-KERNEL-NEXT: %n8 = load i64, i64* %p8, align 8
+; LOAD-KERNEL-NEXT: %4 = bitcast i128* %p16 to i8*
+; LOAD-KERNEL-NEXT: call void @llvm.asan.check.memaccess(i8* %4, i32 9)
+; LOAD-KERNEL-NEXT: %n16 = load i128, i128* %p16, align 16
+  ret void
+}
+
+define void @store(i8* %p1, i16* %p2, i32* %p4, i64* %p8, i128* %p16)
+sanitize_address {
+  store i8 0, i8* %p1, align 1
+  store i16 0, i16* %p2, align 2
+  store i32 0, i32* %p4, align 4
+  store i64 0, i64* %p8, align 8
+  store i128 0, i128* %p16, align 16
+; STORE:      call void @llvm.asan.check.memaccess(i8* %p1, i32 32)
+; STORE-NEXT: store i8 0, i8* %p1, align 1
+; STORE-NEXT: %1 = bitcast i16* %p2 to i8*
+; STORE-NEXT: call void @llvm.asan.check.memaccess(i8* %1, i32 34)
+; STORE-NEXT: store i16 0, i16* %p2, align 2
+; STORE-NEXT: %2 = bitcast i32* %p4 to i8*
+; STORE-NEXT: call void @llvm.asan.check.memaccess(i8* %2, i32 36)
+; STORE-NEXT: store i32 0, i32* %p4, align 4
+; STORE-NEXT: %3 = bitcast i64* %p8 to i8*
+; STORE-NEXT: call void @llvm.asan.check.memaccess(i8* %3, i32 38)
+; STORE-NEXT: store i64 0, i64* %p8, align 8
+; STORE-NEXT: %4 = bitcast i128* %p16 to i8*
+; STORE-NEXT: call void @llvm.asan.check.memaccess(i8* %4, i32 40)
+; STORE-NEXT: store i128 0, i128* %p16, align 16
+
+; STORE-KERNEL:      call void @llvm.asan.check.memaccess(i8* %p1, i32 33)
+; STORE-KERNEL-NEXT: store i8 0, i8* %p1, align 1
+; STORE-KERNEL-NEXT: %1 = bitcast i16* %p2 to i8*
+; STORE-KERNEL-NEXT: call void @llvm.asan.check.memaccess(i8* %1, i32 35)
+; STORE-KERNEL-NEXT: store i16 0, i16* %p2, align 2
+; STORE-KERNEL-NEXT: %2 = bitcast i32* %p4 to i8*
+; STORE-KERNEL-NEXT: call void @llvm.asan.check.memaccess(i8* %2, i32 37)
+; STORE-KERNEL-NEXT: store i32 0, i32* %p4, align 4
+; STORE-KERNEL-NEXT: %3 = bitcast i64* %p8 to i8*
+; STORE-KERNEL-NEXT: call void @llvm.asan.check.memaccess(i8* %3, i32 39)
+; STORE-KERNEL-NEXT: store i64 0, i64* %p8, align 8
+; STORE-KERNEL-NEXT: %4 = bitcast i128* %p16 to i8*
+; STORE-KERNEL-NEXT: call void @llvm.asan.check.memaccess(i8* %4, i32 41)
+; STORE-KERNEL-NEXT: store i128 0, i128* %p16, align 16
+; STORE-KERNEL-NEXT: ret void
+  ret void
+}