[AMDGPU] Consider loads from flat addrspace to be potentially divergent

[scott.linder]

Summary

Loads through flat addrspace to non-uniform values were being marked uniform. In general we can't assume flat loads are uniform, and cases where we can prove they are should be handled through infer-address-spaces.

[arsenm]

lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp

556

on pervious line

test/CodeGen/AMDGPU/divergent-flat.ll

1–2

Test can be a lot simpler and put in test/Analysis/DivergenceAnalysis/AMDGPU with the rest

[scott.linder]

Run through clang-format; simplify and move test case.

[arsenm]

test/Analysis/DivergenceAnalysis/AMDGPU/divergent-flat.ll

5–12 ↗

(On Diff #161552)

You don't need the addrspacecast, and it would be better to just have the argument be flat in the first place. This theoretically could be smarter and say yes in this case

5–12 ↗

(On Diff #161552)

Also it looks like we are missing a test for the private case anyway. Can you just rename the test to loads and add a test for private as well?

[scott.linder]

Simplify test further and add a test for load from scratch.

[arsenm]

LGTM

lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp

 }
 
 /// \returns true if the result of the value could potentially be
 /// different across workitems in a wavefront.
 bool GCNTTIImpl::isSourceOfDivergence(const Value *V) const {
   if (const Argument *A = dyn_cast<Argument>(V))
     return !isArgPassedInSGPR(A);
 
-  // Loads from the private address space are divergent, because threads
-  // can execute the load instruction with the same inputs and get different
-  // results.
+  // Loads from the private and flat address spaces are divergent, because
+  // threads can execute the load instruction with the same inputs and get
+  // different results.
   //
   // All other loads are not divergent, because if threads issue loads with the
   // same arguments, they will always get the same result.
   if (const LoadInst *Load = dyn_cast<LoadInst>(V))
-    return Load->getPointerAddressSpace() == ST->getAMDGPUAS().PRIVATE_ADDRESS;
+    return Load->getPointerAddressSpace() == ST->getAMDGPUAS().PRIVATE_ADDRESS
+      || Load->getPointerAddressSpace() == ST->getAMDGPUAS().FLAT_ADDRESS;
 
   // Atomics are divergent because they are executed sequentially: when an
   // atomic operation refers to the same address in each thread, then each
   // thread after the first sees the value written by the previous thread as
   // original value.
   if (isa<AtomicRMWInst>(V) || isa<AtomicCmpXchgInst>(V))
     return true;
 

test/CodeGen/AMDGPU/divergent-flat.ll

+; RUN: llc -march=amdgcn -mcpu=gfx900 -o - %s | FileCheck %s --check-prefix=ASM
+; RUN: llc -march=amdgcn -mcpu=gfx900 -stop-after=structurizecfg -o - %s | FileCheck %s --check-prefix=STRUCTURIZECFG
+
+; Test that we do not consider loads from flat addrspace to be uniform.
+
+define amdgpu_kernel void @spam(float* %a) #0 {
+  %priv = alloca i32, align 4, addrspace(5)
+  %flat = addrspacecast i32 addrspace(5)* %priv to i32*
+  %idx = call i32 @llvm.amdgcn.workitem.id.x()
+
+  store i32 %idx, i32* %flat, align 4
+  %b = load i32, i32* %flat, align 4
+
+  %cmp = icmp slt i32 %b, 1
+; ASM: s_mov_b64 exec, s[{{[0-9]+}}:{{[0-9]+}}]
+; ASM-NOT: s_cbranch_vccnz
+; STRUCTURIZECFG-NOT: structurizecfg.uniform
+  br i1 %cmp, label %body, label %end
+
+body:
+  store float 1.000000e+00, float* %a, align 4
+  br label %end
+
+end:
+  ret void
+}
+
+declare i32 @llvm.amdgcn.workitem.id.x()
+
+attributes #0 = { noinline optnone }