AMDGPU/SI: Fix s_waitcnt insertion for flat instructions

[• tstellarAMD]

Summary

This was broken in r260694 which swapped the address and data operands
for flat store instructions. The code in SIInsertWaits assumes
that the data operand always comes before the address operand, so
we need to add a special case for flat.

[arsenm]

LGTM. A test seems hard, but do you have a small reduced testcase?

[• tstellarAMD]

In D17366#355350, @arsenm wrote:

LGTM. A test seems hard, but do you have a small reduced testcase?

No, one of the C++AMP test hit this and the test case is really huge. I can try to write a small one.

llvm/trunk/lib/Target/AMDGPU/SIInsertWaits.cpp

bool SIInsertWaits::isOpRelevant(MachineOperand &Op) {
   if (MI.getOpcode() == AMDGPU::EXP)
     return true;
 
   // For stores the stored value is also relevant
   if (!MI.getDesc().mayStore())
     return false;
 
   // Check if this operand is the value being stored.
-  // Special case for DS instructions, since the address
+  // Special case for DS/FLAT instructions, since the address
   // operand comes before the value operand and it may have
   // multiple data operands.
 
-  if (TII->isDS(MI)) {
+  if (TII->isDS(MI) || TII->isFLAT(MI)) {
     MachineOperand *Data = TII->getNamedOperand(MI, AMDGPU::OpName::data);
     if (Data && Op.isIdenticalTo(*Data))
       return true;
+  }
 
+  if (TII->isDS(MI)) {
     MachineOperand *Data0 = TII->getNamedOperand(MI, AMDGPU::OpName::data0);
     if (Data0 && Op.isIdenticalTo(*Data0))
       return true;
 
     MachineOperand *Data1 = TII->getNamedOperand(MI, AMDGPU::OpName::data1);
     if (Data1 && Op.isIdenticalTo(*Data1))
       return true;
 

llvm/trunk/test/CodeGen/AMDGPU/waitcnt-flat.ll

+; RUN: llc < %s -mtriple=amdgcn--amdhsa -mcpu=kaveri | FileCheck --check-prefix=GCN %s
+; RUN: llc < %s -mtriple=amdgcn--amdhsa -mcpu=fiji | FileCheck --check-prefix=GCN %s
+
+; If flat_store_dword and flat_load_dword use different registers for the data
+; operand, this test is not broken.  It just means it is no longer testing
+; for the original bug.
+
+; GCN: {{^}}test:
+; GCN: flat_store_dword v[{{[0-9]+:[0-9]+}}], [[DATA:v[0-9]+]]
+; GCN: s_waitcnt vmcnt(0) lgkmcnt(0)
+; GCN: flat_load_dword [[DATA]], v[{{[0-9]+:[0-9]+}}]
+define void @test(i32 addrspace(1)* %out, i32 %in) {
+  store volatile i32 0, i32 addrspace(1)* %out
+  %val = load volatile i32, i32 addrspace(1)* %out
+  ret void
+}