[ASAN][AMDGPU] Add support for accesses to global and constant addrspaces

[rksharma]

Summary

TL;DR This patch adds address sanitizer instrumentation support for accesses to global and constant address spaces in AMDGPU. It strictly avoids instrumenting the stack and assumes x86 as the host.

This patch adds address sanitizer instrumentations to global accesses (addrspace 1 and 4) for the device/GPU code. These addresses are accessible from both the host/CPU and the device/GPU therefore we use the static shadow mapping from the host. For addresses explicitly in addrspace 1 and 4, we can follow the exact same instrumentation that is done for the host.
There are generic/flat addresses (addrspace 0) that can point to global memory (addrspace 1 and 4). We add runtime calls (llvm.amdgcn.is.shared and llvm.amdgcn.is.private) to check if the flat address is a global address at runtime. If it is then we instrument it as we would do for explicit accesses to addrspace 1 or 4.

[rksharma]

clang-formatted

[vitalybuka]

Could you please upload with "arc" tool so we can see context here?

[vitalybuka]

Can you please point to some high level overview how this should work?
If it does not exists, could you please describe that in summary?

[rksharma]

Thanks! Added missing context and tried to explicitly state what we are trying to do in the description. I can make it more verbose if required. Please let me know if I can clarify or add something.

[vitalybuka]

LGTM with few minor comments

@eugenis and @kda mostly FYI

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

1725–1733

can you please move more AMD specific code into instrumentAMDGPUGenericAddress?

1901

Can you please extract all these AddressSpace checks into some functions with self explaining names?

maybe isUnsupportedAmdAddressSpace()
same for 3,5

2426–2429

can you sent this as a separate NFC patch?

llvm/lib/Transforms/Utils/ModuleUtils.cpp

92 ↗

(On Diff #332928)

can you sent this as a separate NFC patch?
does it affect any existing tests?

[rksharma]

Thanks for the review, I've addressed your comments, I've moved more AMDGPU specific code out of instrumentAddress and added a function to check unsupported address spaces.

[rksharma]

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

1725–1733

I tried and moved some more code out of instrumentAddress

1901

I've added one for 3 and 5 which are unsupported

[vitalybuka]

Can you please clang-format the diff?

[vitalybuka]

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

1400–1401

Why can we use isUnsupportedAMDGPUAddrspacehere(PtrTy)?

1705

And the rest of the patch.

1715

Please collect getOrInsertFunction in AddressSanitizer::initializeCallbacks

[vitalybuka]

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

953–964

[rksharma]

Addressed the review comments.

[rksharma]

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

177–178

Is it fine that I've created variables for them or shall I directly use function names as string inside getOrInsertFunction?

1400–1401

We can use isUnsupported but we can not get rid of line:1395 because asan needs to check if the address is in a non-zero addrspace for non-amdgpu targets.

[vitalybuka]

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

177–178

variable lgtm

1330–1336

1699–1701

1705

"." after comments is not done?

1900

[rksharma]

Updated all the comments in the patch, I'm sorry, initially, I misunderstood your review comment to be about something else. Thanks for pointing it out.

[nickdesaulniers]

Are declarations for @llvm.amdgcn.is.shared supposed to be showing up in my IR, even when my target triple is _not_ amdgcn-amd?

[b-sumner]

In D99071#3652434, @nickdesaulniers wrote:

Are declarations for @llvm.amdgcn.is.shared supposed to be showing up in my IR, even when my target triple is _not_ amdgcn-amd?

That's surprising. What is your triple? What targets have you built for?

[pvellien]

In D99071#3652434, @nickdesaulniers wrote:

Are declarations for @llvm.amdgcn.is.shared supposed to be showing up in my IR, even when my target triple is _not_ amdgcn-amd?

@nickdesaulniers thanks for letting us know about this, I would like to understand more about this. Is it possible for you to share a simple test case (along with which target) and build script to me so that I can reproduce it on my end.

[nickdesaulniers]

In D99071#3652570, @pvellien wrote:

In D99071#3652434, @nickdesaulniers wrote:

Are declarations for @llvm.amdgcn.is.shared supposed to be showing up in my IR, even when my target triple is _not_ amdgcn-amd?

@nickdesaulniers thanks for letting us know about this, I would like to understand more about this. Is it possible for you to share a simple test case (along with which target) and build script to me so that I can reproduce it on my end.

$ cd llvm-project
$ clang -cc1 -internal-isystem llvm/build/lib/clang/15.0.0/include -nostdsysteminc -std=gnu2x -triple x86_64-linux-gnu clang/test/CodeGen/attr-function-return.c -emit-llvm -o - -mfunction-return=thunk-extern -fsanitize=address

[pvellien]

In D99071#3652603, @nickdesaulniers wrote:

In D99071#3652570, @pvellien wrote:

In D99071#3652434, @nickdesaulniers wrote:

Are declarations for @llvm.amdgcn.is.shared supposed to be showing up in my IR, even when my target triple is _not_ amdgcn-amd?

@nickdesaulniers thanks for letting us know about this, I would like to understand more about this. Is it possible for you to share a simple test case (along with which target) and build script to me so that I can reproduce it on my end.

$ cd llvm-project
$ clang -cc1 -internal-isystem llvm/build/lib/clang/15.0.0/include -nostdsysteminc -std=gnu2x -triple x86_64-linux-gnu clang/test/CodeGen/attr-function-return.c -emit-llvm -o - -mfunction-return=thunk-extern -fsanitize=address

Ok thanks

llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp

const char kAsanOptionDetectUseAfterReturn[] =
     "__asan_option_detect_stack_use_after_return";
 
 const char kAsanShadowMemoryDynamicAddress[] =
     "__asan_shadow_memory_dynamic_address";
 
 const char kAsanAllocaPoison[] = "__asan_alloca_poison";
 const char kAsanAllocasUnpoison[] = "__asan_allocas_unpoison";
 
+const char kAMDGPUAddressSharedName[] = "llvm.amdgcn.is.shared";
+const char kAMDGPUAddressPrivateName[] = "llvm.amdgcn.is.private";
+
 // Accesses sizes are powers of two: 1, 2, 4, 8, 16.
 static const size_t kNumberOfAccessSizes = 5;
 
 static const unsigned kAllocaRzSize = 32;
 
 // Command-line flags.
 
 static cl::opt<bool> ClEnableKasan(
static ShadowMapping getShadowMapping(Triple &TargetTriple, int LongSize,
   bool IsMIPS64 = TargetTriple.isMIPS64();
   bool IsArmOrThumb = TargetTriple.isARM() || TargetTriple.isThumb();
   bool IsAArch64 = TargetTriple.getArch() == Triple::aarch64;
   bool IsRISCV64 = TargetTriple.getArch() == Triple::riscv64;
   bool IsWindows = TargetTriple.isOSWindows();
   bool IsFuchsia = TargetTriple.isOSFuchsia();
   bool IsMyriad = TargetTriple.getVendor() == llvm::Triple::Myriad;
   bool IsEmscripten = TargetTriple.isOSEmscripten();
+  bool IsAMDGPU = TargetTriple.isAMDGPU();
+
+  // Asan support for AMDGPU assumes X86 as the host right now.
+  if (IsAMDGPU)
+    IsX86_64 = true;
 
   ShadowMapping Mapping;
 
   Mapping.Scale = IsMyriad ? kMyriadShadowScale : kDefaultShadowScale;
   if (ClMappingScale.getNumOccurrences() > 0) {
     Mapping.Scale = ClMappingScale;
   }
 
struct AddressSanitizer {
 
   void instrumentMop(ObjectSizeOffsetVisitor &ObjSizeVis,
                      InterestingMemoryOperand &O, bool UseCalls,
                      const DataLayout &DL);
   void instrumentPointerComparisonOrSubtraction(Instruction *I);
   void instrumentAddress(Instruction *OrigIns, Instruction *InsertBefore,
                          Value *Addr, uint32_t TypeSize, bool IsWrite,
                          Value *SizeArgument, bool UseCalls, uint32_t Exp);
+  Instruction *instrumentAMDGPUAddress(Instruction *OrigIns,
+                                       Instruction *InsertBefore, Value *Addr,
+                                       uint32_t TypeSize, bool IsWrite,
+                                       Value *SizeArgument);
   void instrumentUnusualSizeOrAlignment(Instruction *I,
                                         Instruction *InsertBefore, Value *Addr,
                                         uint32_t TypeSize, bool IsWrite,
                                         Value *SizeArgument, bool UseCalls,
                                         uint32_t Exp);
   Value *createSlowPathCmp(IRBuilder<> &IRB, Value *AddrLong,
                            Value *ShadowValue, uint32_t TypeSize);
   Instruction *generateCrashCode(Instruction *InsertBefore, Value *Addr,
private:
   // These arrays is indexed by AccessIsWrite and Experiment.
   FunctionCallee AsanErrorCallbackSized[2][2];
   FunctionCallee AsanMemoryAccessCallbackSized[2][2];
 
   FunctionCallee AsanMemmove, AsanMemcpy, AsanMemset;
   Value *LocalDynamicShadow = nullptr;
   const GlobalsMetadata &GlobalsMD;
   DenseMap<const AllocaInst *, bool> ProcessedAllocas;
+
+  FunctionCallee AMDGPUAddressShared;
+  FunctionCallee AMDGPUAddressPrivate;
 };
 
 class AddressSanitizerLegacyPass : public FunctionPass {
 public:
   static char ID;
 
   explicit AddressSanitizerLegacyPass(bool CompileKernel = false,
                                       bool Recover = false,
struct FunctionStackPoisoner : public InstVisitor<FunctionStackPoisoner> {
 
   SmallVector<AllocaInst *, 1> DynamicAllocaVec;
   SmallVector<IntrinsicInst *, 1> StackRestoreVec;
   AllocaInst *DynamicAllocaLayout = nullptr;
   IntrinsicInst *LocalEscapeCall = nullptr;
 
   bool HasInlineAsm = false;
   bool HasReturnsTwiceCall = false;
+  bool PoisonStack;
 
   FunctionStackPoisoner(Function &F, AddressSanitizer &ASan)
       : F(F), ASan(ASan), DIB(*F.getParent(), /*AllowUnresolved*/ false),
         C(ASan.C), IntptrTy(ASan.IntptrTy),
         IntptrPtrTy(PointerType::get(IntptrTy, 0)), Mapping(ASan.Mapping),
-        StackAlignment(1 << Mapping.Scale) {}
+        StackAlignment(1 << Mapping.Scale),
+        PoisonStack(ClStack &&
+                    !Triple(F.getParent()->getTargetTriple()).isAMDGPU()) {}
 
   bool runOnFunction() {
-    if (!ClStack) return false;
+    if (!PoisonStack)
+      return false;
 
     if (ClRedzoneByvalArgs)
       copyArgsPassedByValToAllocas();
 
     // Collect alloca, ret, lifetime instructions etc.
     for (BasicBlock *BB : depth_first(&F.getEntryBlock())) visit(*BB);
 
     if (AllocaVec.empty() && DynamicAllocaVec.empty()) return false;
 
static bool GlobalWasGeneratedByCompiler(GlobalVariable *G) {
 
   // Do not instrument gcov counter arrays.
   if (G->getName() == "__llvm_gcov_ctr")
     return true;
 
   return false;
 }
 
+static bool isUnsupportedAMDGPUAddrspace(Value *Addr) {
+  Type *PtrTy = cast<PointerType>(Addr->getType()->getScalarType());
+  unsigned int AddrSpace = PtrTy->getPointerAddressSpace();
+  if (AddrSpace == 3 || AddrSpace == 5)
+    return true;
+  return false;
+}
+
 Value *AddressSanitizer::memToShadow(Value *Shadow, IRBuilder<> &IRB) {
   // Shadow >> scale
   Shadow = IRB.CreateLShr(Shadow, Mapping.Scale);
   if (Mapping.Offset == 0) return Shadow;
   // (Shadow >> scale) | offset
   Value *ShadowBase;
   if (LocalDynamicShadow)
     ShadowBase = LocalDynamicShadow;
bool IsInteresting =
        // swifterror allocas are register promoted by ISel
        !AI.isSwiftError());
 
   ProcessedAllocas[&AI] = IsInteresting;
   return IsInteresting;
 }
 
 bool AddressSanitizer::ignoreAccess(Value *Ptr) {
-  // Do not instrument acesses from different address spaces; we cannot deal
-  // with them.
+  // Instrument acesses from different address spaces only for AMDGPU.
   Type *PtrTy = cast<PointerType>(Ptr->getType()->getScalarType());
-  if (PtrTy->getPointerAddressSpace() != 0)
+  if (PtrTy->getPointerAddressSpace() != 0 &&
+      !(TargetTriple.isAMDGPU() && !isUnsupportedAMDGPUAddrspace(Ptr)))
     return true;
 
   // Ignore swifterror addresses.
   // swifterror memory addresses are mem2reg promoted by instruction
   // selection. As such they cannot have regular uses like an instrumentation
   // function and it makes no sense to track them as memory.
   if (Ptr->isSwiftError())
     return true;
LastAccessedByte = IRB.CreateAdd(
         LastAccessedByte, ConstantInt::get(IntptrTy, TypeSize / 8 - 1));
   // (uint8_t) ((Addr & (Granularity-1)) + size - 1)
   LastAccessedByte =
       IRB.CreateIntCast(LastAccessedByte, ShadowValue->getType(), false);
   // ((uint8_t) ((Addr & (Granularity-1)) + size - 1)) >= ShadowValue
   return IRB.CreateICmpSGE(LastAccessedByte, ShadowValue);
 }
 
+Instruction *AddressSanitizer::instrumentAMDGPUAddress(
+    Instruction *OrigIns, Instruction *InsertBefore, Value *Addr,
+    uint32_t TypeSize, bool IsWrite, Value *SizeArgument) {
+  // Do not instrument unsupported addrspaces.
+  if (isUnsupportedAMDGPUAddrspace(Addr))
+    return nullptr;
+  Type *PtrTy = cast<PointerType>(Addr->getType()->getScalarType());
+  // Follow host instrumentation for global and constant addresses.
+  if (PtrTy->getPointerAddressSpace() != 0)
+    return InsertBefore;
+  // Instrument generic addresses in supported addressspaces.
+  IRBuilder<> IRB(InsertBefore);
+  Value *AddrLong = IRB.CreatePointerCast(Addr, IRB.getInt8PtrTy());
+  Value *IsShared = IRB.CreateCall(AMDGPUAddressShared, {AddrLong});
+  Value *IsPrivate = IRB.CreateCall(AMDGPUAddressPrivate, {AddrLong});
+  Value *IsSharedOrPrivate = IRB.CreateOr(IsShared, IsPrivate);
+  Value *Cmp = IRB.CreateICmpNE(IRB.getTrue(), IsSharedOrPrivate);
+  Value *AddrSpaceZeroLanding =
+      SplitBlockAndInsertIfThen(Cmp, InsertBefore, false);
+  InsertBefore = cast<Instruction>(AddrSpaceZeroLanding);
+  return InsertBefore;
+}
+
 void AddressSanitizer::instrumentAddress(Instruction *OrigIns,
                                          Instruction *InsertBefore, Value *Addr,
                                          uint32_t TypeSize, bool IsWrite,
                                          Value *SizeArgument, bool UseCalls,
                                          uint32_t Exp) {
   bool IsMyriad = TargetTriple.getVendor() == llvm::Triple::Myriad;
 
+  if (TargetTriple.isAMDGPU()) {
+    InsertBefore = instrumentAMDGPUAddress(OrigIns, InsertBefore, Addr,
+                                           TypeSize, IsWrite, SizeArgument);
+    if (!InsertBefore)
+      return;
+  }
+
   IRBuilder<> IRB(InsertBefore);
   Value *AddrLong = IRB.CreatePointerCast(Addr, IntptrTy);
   size_t AccessSizeIndex = TypeSizeToSizeIndex(TypeSize);
 
   if (UseCalls) {
     if (Exp == 0)
       IRB.CreateCall(AsanMemoryAccessCallback[IsWrite][0][AccessSizeIndex],
                      AddrLong);
     else
       IRB.CreateCall(AsanMemoryAccessCallback[IsWrite][1][AccessSizeIndex],
bool ModuleAddressSanitizer::shouldInstrumentGlobal(GlobalVariable *G) const {
   Type *Ty = G->getValueType();
   LLVM_DEBUG(dbgs() << "GLOBAL: " << *G << "\n");
 
   // FIXME: Metadata should be attched directly to the global directly instead
   // of being added to llvm.asan.globals.
   if (GlobalsMD.get(G).IsExcluded) return false;
   if (!Ty->isSized()) return false;
   if (!G->hasInitializer()) return false;
-  // Only instrument globals of default address spaces
-  if (G->getAddressSpace()) return false;
+  // Globals in address space 1 and 4 are supported for AMDGPU.
+  if (G->getAddressSpace() &&
+      !(TargetTriple.isAMDGPU() && !isUnsupportedAMDGPUAddrspace(G)))
+    return false;
   if (GlobalWasGeneratedByCompiler(G)) return false; // Our own globals.
   // Two problems with thread-locals:
   //   - The address of the main thread's copy can't be computed at link-time.
   //   - Need to poison all copies, not just the main thread's one.
   if (G->isThreadLocal()) return false;
   // For now, just ignore this Global if the alignment is large.
   if (G->getAlignment() > getMinRedzoneSizeForGlobal()) return false;
 
for (size_t i = 0; i < n; i++) {
     StructType *NewTy = StructType::get(Ty, RightRedZoneTy);
     Constant *NewInitializer = ConstantStruct::get(
         NewTy, G->getInitializer(), Constant::getNullValue(RightRedZoneTy));
 
     // Create a new global variable with enough space for a redzone.
     GlobalValue::LinkageTypes Linkage = G->getLinkage();
     if (G->isConstant() && Linkage == GlobalValue::PrivateLinkage)
       Linkage = GlobalValue::InternalLinkage;
     GlobalVariable *NewGlobal = new GlobalVariable(
         M, NewTy, G->isConstant(), Linkage, NewInitializer, "", G,
         G->getThreadLocalMode(), G->getAddressSpace());
     NewGlobal->copyAttributesFrom(G);
     NewGlobal->setComdat(G->getComdat());
     NewGlobal->setAlignment(MaybeAlign(getMinRedzoneSizeForGlobal()));
     // Don't fold globals with redzones. ODR violation detector and redzone
     // poisoning implicitly creates a dependence on the global's address, so it
     // is no longer valid for it to be marked unnamed_addr.
     NewGlobal->setUnnamedAddr(GlobalValue::UnnamedAddr::None);
 
     // Move null-terminated C strings to "__asan_cstring" section on Darwin.
void AddressSanitizer::initializeCallbacks(Module &M) {
 
   AsanPtrCmpFunction =
       M.getOrInsertFunction(kAsanPtrCmp, IRB.getVoidTy(), IntptrTy, IntptrTy);
   AsanPtrSubFunction =
       M.getOrInsertFunction(kAsanPtrSub, IRB.getVoidTy(), IntptrTy, IntptrTy);
   if (Mapping.InGlobal)
     AsanShadowGlobal = M.getOrInsertGlobal("__asan_shadow",
                                            ArrayType::get(IRB.getInt8Ty(), 0));
+
+  AMDGPUAddressShared = M.getOrInsertFunction(
+      kAMDGPUAddressSharedName, IRB.getInt1Ty(), IRB.getInt8PtrTy());
+  AMDGPUAddressPrivate = M.getOrInsertFunction(
+      kAMDGPUAddressPrivateName, IRB.getInt1Ty(), IRB.getInt8PtrTy());
 }
 
 bool AddressSanitizer::maybeInsertAsanInitAtFunctionEntry(Function &F) {
   // For each NSObject descendant having a +load method, this method is invoked
   // by the ObjC runtime before any of the static constructors is called.
   // Therefore we need to instrument such methods with a call to __asan_init
   // at the beginning in order to initialize our runtime before any access to
   // the shadow memory.

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/adaptive_constant_global_redzones.ll

+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -S | FileCheck %s
+target datalayout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7"
+target triple = "amdgcn-amd-amdhsa"
+
+; Here we check that the global redzone sizes grow with the object size
+; for objects in constant address space.
+
+@G10 = addrspace(4) global [10 x i8] zeroinitializer, align 1
+; CHECK: @G10 = addrspace(4) global { [10 x i8], [54 x i8] }
+
+@G31 = addrspace(4) global [31 x i8] zeroinitializer, align 1
+@G32 = addrspace(4) global [32 x i8] zeroinitializer, align 1
+@G33 = addrspace(4) global [33 x i8] zeroinitializer, align 1
+; CHECK: @G31 = addrspace(4) global { [31 x i8], [33 x i8] }
+; CHECK: @G32 = addrspace(4) global { [32 x i8], [32 x i8] }
+; CHECK: @G33 = addrspace(4) global { [33 x i8], [63 x i8] }
+
+@G63 = addrspace(4) global [63 x i8] zeroinitializer, align 1
+@G64 = addrspace(4) global [64 x i8] zeroinitializer, align 1
+@G65 = addrspace(4) global [65 x i8] zeroinitializer, align 1
+; CHECK: @G63 = addrspace(4) global { [63 x i8], [33 x i8] }
+; CHECK: @G64 = addrspace(4) global { [64 x i8], [32 x i8] }
+; CHECK: @G65 = addrspace(4) global { [65 x i8], [63 x i8] }
+
+@G127 = addrspace(4) global [127 x i8] zeroinitializer, align 1
+@G128 = addrspace(4) global [128 x i8] zeroinitializer, align 1
+@G129 = addrspace(4) global [129 x i8] zeroinitializer, align 1
+; CHECK: @G127 = addrspace(4) global { [127 x i8], [33 x i8] }
+; CHECK: @G128 = addrspace(4) global { [128 x i8], [32 x i8] }
+; CHECK: @G129 = addrspace(4) global { [129 x i8], [63 x i8] }
+
+@G255 = addrspace(4) global [255 x i8] zeroinitializer, align 1
+@G256 = addrspace(4) global [256 x i8] zeroinitializer, align 1
+@G257 = addrspace(4) global [257 x i8] zeroinitializer, align 1
+; CHECK: @G255 = addrspace(4) global { [255 x i8], [33 x i8] }
+; CHECK: @G256 = addrspace(4) global { [256 x i8], [64 x i8] }
+; CHECK: @G257 = addrspace(4) global { [257 x i8], [95 x i8] }
+
+@G511 = addrspace(4) global [511 x i8] zeroinitializer, align 1
+@G512 = addrspace(4) global [512 x i8] zeroinitializer, align 1
+@G513 = addrspace(4) global [513 x i8] zeroinitializer, align 1
+; CHECK: @G511 = addrspace(4) global { [511 x i8], [97 x i8] }
+; CHECK: @G512 = addrspace(4) global { [512 x i8], [128 x i8] }
+; CHECK: @G513 = addrspace(4) global { [513 x i8], [159 x i8] }
+
+@G1023 = addrspace(4) global [1023 x i8] zeroinitializer, align 1
+@G1024 = addrspace(4) global [1024 x i8] zeroinitializer, align 1
+@G1025 = addrspace(4) global [1025 x i8] zeroinitializer, align 1
+; CHECK: @G1023 = addrspace(4) global { [1023 x i8], [225 x i8] }
+; CHECK: @G1024 = addrspace(4) global { [1024 x i8], [256 x i8] }
+; CHECK: @G1025 = addrspace(4) global { [1025 x i8], [287 x i8] }
+
+@G1000000 = addrspace(4) global [1000000 x i8] zeroinitializer, align 1
+@G10000000 = addrspace(4) global [10000000 x i8] zeroinitializer, align 1
+@G100000000 = addrspace(4) global [100000000 x i8] zeroinitializer, align 1
+; CHECK: @G1000000 = addrspace(4) global { [1000000 x i8], [249984 x i8] }
+; CHECK: @G10000000 = addrspace(4) global { [10000000 x i8], [262144 x i8] }
+; CHECK: @G100000000 = addrspace(4) global { [100000000 x i8], [262144 x i8] }

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/adaptive_global_redzones.ll

+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -S | FileCheck %s
+target datalayout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7"
+target triple = "amdgcn-amd-amdhsa"
+
+; Here we check that the global redzone sizes grow with the object size
+; for objects in global address space.
+
+@G10 = addrspace(1) global [10 x i8] zeroinitializer, align 1
+; CHECK: @G10 = addrspace(1) global { [10 x i8], [54 x i8] }
+
+@G31 = addrspace(1) global [31 x i8] zeroinitializer, align 1
+@G32 = addrspace(1) global [32 x i8] zeroinitializer, align 1
+@G33 = addrspace(1) global [33 x i8] zeroinitializer, align 1
+; CHECK: @G31 = addrspace(1) global { [31 x i8], [33 x i8] }
+; CHECK: @G32 = addrspace(1) global { [32 x i8], [32 x i8] }
+; CHECK: @G33 = addrspace(1) global { [33 x i8], [63 x i8] }
+
+@G63 = addrspace(1) global [63 x i8] zeroinitializer, align 1
+@G64 = addrspace(1) global [64 x i8] zeroinitializer, align 1
+@G65 = addrspace(1) global [65 x i8] zeroinitializer, align 1
+; CHECK: @G63 = addrspace(1) global { [63 x i8], [33 x i8] }
+; CHECK: @G64 = addrspace(1) global { [64 x i8], [32 x i8] }
+; CHECK: @G65 = addrspace(1) global { [65 x i8], [63 x i8] }
+
+@G127 = addrspace(1) global [127 x i8] zeroinitializer, align 1
+@G128 = addrspace(1) global [128 x i8] zeroinitializer, align 1
+@G129 = addrspace(1) global [129 x i8] zeroinitializer, align 1
+; CHECK: @G127 = addrspace(1) global { [127 x i8], [33 x i8] }
+; CHECK: @G128 = addrspace(1) global { [128 x i8], [32 x i8] }
+; CHECK: @G129 = addrspace(1) global { [129 x i8], [63 x i8] }
+
+@G255 = addrspace(1) global [255 x i8] zeroinitializer, align 1
+@G256 = addrspace(1) global [256 x i8] zeroinitializer, align 1
+@G257 = addrspace(1) global [257 x i8] zeroinitializer, align 1
+; CHECK: @G255 = addrspace(1) global { [255 x i8], [33 x i8] }
+; CHECK: @G256 = addrspace(1) global { [256 x i8], [64 x i8] }
+; CHECK: @G257 = addrspace(1) global { [257 x i8], [95 x i8] }
+
+@G511 = addrspace(1) global [511 x i8] zeroinitializer, align 1
+@G512 = addrspace(1) global [512 x i8] zeroinitializer, align 1
+@G513 = addrspace(1) global [513 x i8] zeroinitializer, align 1
+; CHECK: @G511 = addrspace(1) global { [511 x i8], [97 x i8] }
+; CHECK: @G512 = addrspace(1) global { [512 x i8], [128 x i8] }
+; CHECK: @G513 = addrspace(1) global { [513 x i8], [159 x i8] }
+
+@G1023 = addrspace(1) global [1023 x i8] zeroinitializer, align 1
+@G1024 = addrspace(1) global [1024 x i8] zeroinitializer, align 1
+@G1025 = addrspace(1) global [1025 x i8] zeroinitializer, align 1
+; CHECK: @G1023 = addrspace(1) global { [1023 x i8], [225 x i8] }
+; CHECK: @G1024 = addrspace(1) global { [1024 x i8], [256 x i8] }
+; CHECK: @G1025 = addrspace(1) global { [1025 x i8], [287 x i8] }
+
+@G1000000 = addrspace(1) global [1000000 x i8] zeroinitializer, align 1
+@G10000000 = addrspace(1) global [10000000 x i8] zeroinitializer, align 1
+@G100000000 = addrspace(1) global [100000000 x i8] zeroinitializer, align 1
+; CHECK: @G1000000 = addrspace(1) global { [1000000 x i8], [249984 x i8] }
+; CHECK: @G10000000 = addrspace(1) global { [10000000 x i8], [262144 x i8] }
+; CHECK: @G100000000 = addrspace(1) global { [100000000 x i8], [262144 x i8] }

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/asan_do_not_instrument_lds.ll

+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -S | FileCheck %s
+target datalayout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7"
+target triple = "amdgcn-amd-amdhsa"
+
+; Memory access to lds are not instrumented
+
+@count = addrspace(3) global [100 x i32] undef, align 16
+
+define protected amdgpu_kernel void @lds_store(i32 %i) sanitize_address {
+entry:
+  ; CHECK-NOT: call * __asan_report
+  %arrayidx1 = getelementptr inbounds [100 x i32], [100 x i32] addrspace(3)* @count, i32 0, i32 %i
+  store i32 0, i32 addrspace(3)* %arrayidx1, align 4
+  ret void
+}
+
+define protected amdgpu_kernel void @lds_load(i32 %i) sanitize_address {
+entry:
+  ; CHECK-NOT: call * __asan_report
+  %arrayidx1 = getelementptr inbounds [100 x i32], [100 x i32] addrspace(3)* @count, i32 0, i32 %i
+  %0 = load i32, i32 addrspace(3)* %arrayidx1, align 4
+  ret void
+}

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/asan_do_not_instrument_scratch.ll

+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -S | FileCheck %s
+target datalayout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7"
+target triple = "amdgcn-amd-amdhsa"
+
+; Memory access to scratch are not instrumented
+
+define protected amdgpu_kernel void @scratch_store(i32 %i) sanitize_address {
+entry:
+  ; CHECK-NOT: call * __asan_report
+  %c = alloca i32, align 4, addrspace(5)
+  store i32 0, i32 addrspace(5)* %c, align 4
+  ret void
+}
+
+define protected amdgpu_kernel void @scratch_load(i32 %i) sanitize_address {
+entry:
+  ; CHECK-NOT: call * __asan_report
+  %c = alloca i32, align 4, addrspace(5)
+  %0 = load i32, i32 addrspace(5)* %c, align 4
+  ret void
+}

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/asan_instrument_constant_address_space.ll

+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -S | FileCheck %s
+target datalayout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7"
+target triple = "amdgcn-amd-amdhsa"
+
+@x = addrspace(4) global [2 x i32] zeroinitializer, align 4
+
+define protected amdgpu_kernel void @constant_load(i64 %i) sanitize_address {
+entry:
+; CHECK-LABEL: @constant_load
+; CHECK-NOT: load
+;
+; CHECK:   %[[LOAD_ADDR:[^ ]*]] = ptrtoint i32 addrspace(4)* %a to i64
+; CHECK:   lshr i64 %[[LOAD_ADDR]], 3
+; CHECK:   {{or|add}}
+; CHECK:   %[[LOAD_SHADOW_PTR:[^ ]*]] = inttoptr
+; CHECK:   %[[LOAD_SHADOW:[^ ]*]] = load i8, i8* %[[LOAD_SHADOW_PTR]]
+; CHECK:   icmp ne i8
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; CHECK:   add i64 %{{.*}}, 3
+; CHECK:   trunc i64 %{{.*}} to i8
+; CHECK:   icmp sge i8 %{{.*}}, %[[LOAD_SHADOW]]
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; The crash block reports the error.
+; CHECK:   call void @__asan_report_load4(i64 %[[LOAD_ADDR]])
+; CHECK:   unreachable
+;
+; The actual load.
+; CHECK:   load i32, i32 addrspace(4)* %a
+; CHECK:   ret void
+
+  %a = getelementptr inbounds [2 x i32], [2 x i32]  addrspace(4)* @x, i64 0, i64 %i
+  %q = load i32, i32 addrspace(4)* %a, align 4
+  ret void
+}

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/asan_instrument_generic_address_space.ll

+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -S | FileCheck %s
+target datalayout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7"
+target triple = "amdgcn-amd-amdhsa"
+
+define protected amdgpu_kernel void @generic_store(i32 addrspace(1)* %p, i32 %i) sanitize_address {
+entry:
+; CHECK-LABEL: @generic_store
+; CHECK-NOT: store
+; CHECK:   %[[GENERIC_ADDR:[^ ]*]] = bitcast i32* %q to i8*
+; CHECK:   call i1 @llvm.amdgcn.is.shared(i8* %[[GENERIC_ADDR]])
+; CHECK:   call i1 @llvm.amdgcn.is.private(i8* %[[GENERIC_ADDR]])
+; CHECK:   or
+; CHECK:   icmp ne i1 
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; CHECK:   %[[STORE_ADDR:[^ ]*]] = ptrtoint i32* %q to i64
+; CHECK:   lshr i64 %[[STORE_ADDR]], 3
+; CHECK:   {{or|add}}
+; CHECK:   %[[STORE_SHADOW_PTR:[^ ]*]] = inttoptr
+; CHECK:   %[[STORE_SHADOW:[^ ]*]] = load i8, i8* %[[STORE_SHADOW_PTR]]
+; CHECK:   icmp ne i8
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; CHECK:   add i64 %{{.*}}, 3
+; CHECK:   trunc i64 %{{.*}} to i8
+; CHECK:   icmp sge i8 %{{.*}}, %[[STORE_SHADOW]]
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; The crash block reports the error.
+; CHECK:   call void @__asan_report_store4(i64 %[[STORE_ADDR]])
+; CHECK:   unreachable
+;
+; The actual store.
+; CHECK:   store i32 0, i32* %q
+; CHECK:   ret void
+
+  %q = addrspacecast i32 addrspace(1)* %p to i32*
+  store i32 0, i32* %q, align 4
+  ret void
+}
+
+define protected amdgpu_kernel void @generic_load(i32 addrspace(1)* %p, i32 %i) sanitize_address {
+entry:
+; CHECK-LABEL: @generic_load
+; CHECK-NOT: load
+; CHECK:   %[[GENERIC_ADDR:[^ ]*]] = bitcast i32* %q to i8*
+; CHECK:   call i1 @llvm.amdgcn.is.shared(i8* %[[GENERIC_ADDR]])
+; CHECK:   call i1 @llvm.amdgcn.is.private(i8* %[[GENERIC_ADDR]])
+; CHECK:   or
+; CHECK:   icmp ne i1 
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; CHECK:   %[[STORE_ADDR:[^ ]*]] = ptrtoint i32* %q to i64
+; CHECK:   lshr i64 %[[STORE_ADDR]], 3
+; CHECK:   {{or|add}}
+; CHECK:   %[[STORE_SHADOW_PTR:[^ ]*]] = inttoptr
+; CHECK:   %[[STORE_SHADOW:[^ ]*]] = load i8, i8* %[[STORE_SHADOW_PTR]]
+; CHECK:   icmp ne i8
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; CHECK:   add i64 %{{.*}}, 3
+; CHECK:   trunc i64 %{{.*}} to i8
+; CHECK:   icmp sge i8 %{{.*}}, %[[STORE_SHADOW]]
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; The crash block reports the error.
+; CHECK:   call void @__asan_report_load4(i64 %[[STORE_ADDR]])
+; CHECK:   unreachable
+;
+; The actual store.
+; CHECK:   load i32, i32* %q
+; CHECK:   ret void
+
+  %q = addrspacecast i32 addrspace(1)* %p to i32*
+  %r = load i32, i32* %q, align 4
+  ret void
+}

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/asan_instrument_global_address_space.ll

+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -S | FileCheck %s
+target datalayout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7"
+target triple = "amdgcn-amd-amdhsa"
+
+define protected amdgpu_kernel void @global_store(i32 addrspace(1)* %p, i32 %i) sanitize_address {
+entry:
+; CHECK-LABEL: @global_store
+; CHECK-NOT: store
+;
+; CHECK:   %[[STORE_ADDR:[^ ]*]] = ptrtoint i32 addrspace(1)* %p to i64
+; CHECK:   lshr i64 %[[STORE_ADDR]], 3
+; CHECK:   or
+; CHECK:   %[[STORE_SHADOW_PTR:[^ ]*]] = inttoptr
+; CHECK:   %[[STORE_SHADOW:[^ ]*]] = load i8, i8* %[[STORE_SHADOW_PTR]]
+; CHECK:   icmp ne i8
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; CHECK:   add i64 %{{.*}}, 3
+; CHECK:   trunc i64 %{{.*}} to i8
+; CHECK:   icmp sge i8 %{{.*}}, %[[STORE_SHADOW]]
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; The crash block reports the error.
+; CHECK:   call void @__asan_report_store4(i64 %[[STORE_ADDR]])
+; CHECK:   unreachable
+;
+; The actual store.
+; CHECK:   store i32 0, i32 addrspace(1)* %p
+; CHECK:   ret void
+
+  store i32 0, i32 addrspace(1)* %p, align 4
+  ret void
+}
+
+define protected amdgpu_kernel void @global_load(i32 addrspace(1)* %p, i32 %i) sanitize_address {
+entry:
+; CHECK-LABEL: @global_load
+; CHECK-NOT: load
+;
+; CHECK:   %[[LOAD_ADDR:[^ ]*]] = ptrtoint i32 addrspace(1)* %p to i64
+; CHECK:   lshr i64 %[[LOAD_ADDR]], 3
+; CHECK:   {{add|or}}
+; CHECK:   %[[LOAD_SHADOW_PTR:[^ ]*]] = inttoptr
+; CHECK:   %[[LOAD_SHADOW:[^ ]*]] = load i8, i8* %[[LOAD_SHADOW_PTR]]
+; CHECK:   icmp ne i8
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; CHECK:   add i64 %{{.*}}, 3
+; CHECK:   trunc i64 %{{.*}} to i8
+; CHECK:   icmp sge i8 %{{.*}}, %[[LOAD_SHADOW]]
+; CHECK:   br i1 %{{.*}}, label %{{.*}}, label %{{.*}}
+;
+; The crash block reports the error.
+; CHECK:   call void @__asan_report_load4(i64 %[[LOAD_ADDR]])
+; CHECK:   unreachable
+;
+; The actual load.
+; CHECK:   load i32, i32 addrspace(1)* %p
+; CHECK:   ret void
+
+  %q = load i32, i32 addrspace(1)* %p, align 4
+  ret void
+}

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/global_metadata_addrspacecasts.ll

+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -S | FileCheck %s
+target datalayout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7"
+target triple = "amdgcn-amd-amdhsa"
+
+@g = addrspace(1) global [1 x i32] zeroinitializer, align 4
+
+;CHECK: llvm.asan.globals
+
+!llvm.asan.globals = !{!0, !1}
+!0 = !{[1 x i32] addrspace(1)* @g, null, !"name", i1 false, i1 false}
+!1 = !{i8* addrspacecast (i8 addrspace(1)* bitcast ( [1 x i32] addrspace(1)* @g to i8 addrspace(1)*) to  i8*), null, !"name", i1 false, i1 false}

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/instrument-stack.ll

+; This test checks that we are not instrumenting direct inbound stack accesses.
+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -asan-opt-stack -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -asan-opt-stack -S | FileCheck %s
+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -asan-opt-stack -asan-mapping-scale=5 -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -asan-opt-stack -asan-mapping-scale=5 -S | FileCheck %s
+
+target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+;@sink = global i32* null, align 4
+
+; Ignore direct inbounds stack access.
+define void @foo() uwtable sanitize_address {
+entry:
+  %a = alloca i32, align 4
+  store i32 42, i32* %a, align 4
+  ret void
+; CHECK-LABEL: define void @foo
+; CHECK-NOT: __asan_report
+; CHECK: ret void
+}
+
+; Don't ignore dynamic indexing.
+define void @baz(i64 %i) sanitize_address {
+entry:
+  %a = alloca [10 x i32], align 4
+  %e = getelementptr inbounds [10 x i32], [10 x i32]* %a, i32 0, i64 %i
+  store i32 42, i32* %e, align 4
+  ret void
+; CHECK-LABEL: define void @baz
+; CHECK: __asan_report
+; CHECK: ret void
+}
+
+define void @bar() sanitize_address {
+entry:
+  %a = alloca [10 x i32], align 4
+  %e = getelementptr inbounds [10 x i32], [10 x i32]* %a, i32 0, i64 12
+  store i32 42, i32* %e, align 4
+  ret void
+; CHECK-LABEL: define void @bar
+; CHECK: __asan_report
+; CHECK: ret void
+}
+
+define void @endoftests() sanitize_address {
+entry:
+  ret void
+; CHECK-LABEL: define void @endoftests
+}
+

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/no_redzones_in_lds_globals.ll

+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -S | FileCheck %s
+target datalayout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7"
+target triple = "amdgcn-amd-amdhsa"
+
+@G10 = addrspace(3) global [10 x i8] zeroinitializer, align 1
+; CHECK-NOT: @G10 = addrspace(3) global { [10 x i8], [* x i8] }

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/no_redzones_in_scratch_globals.ll

+; RUN: opt < %s -asan -asan-module -enable-new-pm=0 -S | FileCheck %s
+; RUN: opt < %s -passes='asan-pipeline' -S | FileCheck %s
+target datalayout = "e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7"
+target triple = "amdgcn-amd-amdhsa"
+
+@G10 = addrspace(5) global [10 x i8] zeroinitializer, align 1
+; CHECK-NOT: @G10 = addrspace(5) global { [10 x i8], [* x i8] }