Transform illegal intrinsics to V_ILLEGAL

[Leonc]

Summary

Related tasks:

• SWDEV-240194
• SWDEV-309417
• SWDEV-334876

[b-sumner]

I think this is too focused. There are other image_sample_lz intrinsics and all of them potentially need to be replaced if following this approach.

[rampitec]

There is also global isel.

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

7852

!ST->hasExtendedImageInsts() instead of target check.

[Leonc]

• Updates toward catching and handling all illegal intrinsics.

[Leonc]

• Revert rebase error.

[Leonc]

Next steps:

• Get feedback from team.
• Add tests.
• Extend coverage to INTRINSIC_WO_CHAIN and INTRINSIC_VOID.
• Investigate test failure on gfx940.
  • CodeGenOpenCL/builtins-fp-atomics-gfx940.cl.

Issues:

• Not all intrinsics marked as custom are handled by the target lowering functions. This is the reason INTRINSIC_WO_CHAIN and INTRINSIC_VOID are not covered yet.
  • It might be possible to determine if an intrinsic will be expanded if it is allowed to fall through.

[rampitec]

Missing tests.

[arsenm]

llvm/lib/Target/AMDGPU/AMDGPUISelDAGToDAG.cpp

327 ↗

(On Diff #436108)

Just replace with undef regardless of the type. You also should have just done this when the node was initially created, no need to use the DAG preprocess

llvm/lib/Target/AMDGPU/AMDGPUISelLowering.h

539 ↗

(On Diff #436108)

I don't think it's worth introducing a wrapper node for this

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

7863

You can select directly to the instruction with getMachineNode, no need for the intermediate AMDGPUISD::ILLEGAL

llvm/lib/Target/AMDGPU/SIInstructions.td

3364–3368

This is more complicated. It's actually defined on gfx10 (and I believe can also be encoded as both 32 and 64-bit). For gfx6-9, all 0s does have an interpretation as an almost valid instruction. It decodes fine but violates the constant bus restriction. I would prefer to define a separate V_ILLEGAL that uses all 1s pre gfx10

[Leonc]

llvm/lib/Target/AMDGPU/AMDGPUISelDAGToDAG.cpp

327 ↗

(On Diff #436108)

I did try doing this in TargetLowering originally, but any SDValue returned from a lowering function must have the same type(s) as the node it's replacing. I think this was the same for undef but I'll try it again just to be sure.

[foad]

Where's the rationale? Surely unsupported intrinsics should fail to select, and perhaps be diagnosed even earlier?

[Leonc]

In D123693#3580812, @foad wrote:

Where's the rationale? Surely unsupported intrinsics should fail to select, and perhaps be diagnosed even earlier?

Normally yes. The issue is our libraries contain code for all targets. We rely on dead code elimination to remove illegal intrinsics, but that doesn't happen on -O0.

[foad]

The issue is our libraries contain code for all targets. We rely on dead code elimination to remove illegal intrinsics, but that doesn't happen on -O0.

Why can't you put the code for different subtargets in different functions, each with an appropriate "target-cpu"= attribute?

[Leonc]

In D123693#3580902, @foad wrote:

The issue is our libraries contain code for all targets. We rely on dead code elimination to remove illegal intrinsics, but that doesn't happen on -O0.

Why can't you put the code for different subtargets in different functions, each with an appropriate "target-cpu"= attribute?

That's what I would do. I think there's some resistance to it though, and it would probably require a lot more work.

[foad]

In D123693#3580924, @Leonc wrote:

In D123693#3580902, @foad wrote:

The issue is our libraries contain code for all targets. We rely on dead code elimination to remove illegal intrinsics, but that doesn't happen on -O0.

Why can't you put the code for different subtargets in different functions, each with an appropriate "target-cpu"= attribute?

That's what I would do. I think there's some resistance to it though, and it would probably require a lot more work.

Well I am resistant to selecting intrinsics on subtargets that don't support them :)

How far are you planning to go with this approach? Do you expect to extend it to all AMDGPU intrinsics?

[Leonc]

In D123693#3580962, @foad wrote:

In D123693#3580924, @Leonc wrote:

In D123693#3580902, @foad wrote:

The issue is our libraries contain code for all targets. We rely on dead code elimination to remove illegal intrinsics, but that doesn't happen on -O0.

Why can't you put the code for different subtargets in different functions, each with an appropriate "target-cpu"= attribute?

That's what I would do. I think there's some resistance to it though, and it would probably require a lot more work.

Well I am resistant to selecting intrinsics on subtargets that don't support them :)

How far are you planning to go with this approach? Do you expect to extend it to all AMDGPU intrinsics?

Agreed.

My initial approach was rejected for being too specific. Discussions since then have been about finding a solution for all illegal intrinsic calls. That's the plan unless I hear otherwise.

[bcahoon]

The rationale is for library code that looks like:
void example(float *p, float v) {

if (_ISA_verison == 9008 || __ISA_verison == 9010)
   global_atomic_fadd(p, v);
else
   generic_atomic_fadd(p, v);

}

At optimization levels > 0, the dead code is eliminated, but at optimization level 0, the dead code remain and the compiler attempts to generate an instruction. That instruction would never be executed due to the condition, but it still needs to be generated. In these cases, we don't have multilibs (target specific libraries).

[b-sumner]

Thanks @bcahoon. If there were another available approach that always worked, we would use it.

[dmikushin]

Thank you for working on this. Debug build of PyTorch (BUILD_DEBUG_INFO=1) is also affected. Here is a creduce'd code which crashes rocm5.1.1 clang that you could possibly inciude into the test cases (compile with -O0):

extern "C" __attribute__((device)) void __ockl_atomic_add_noret_f32(float *,
                                                                    float);
__attribute__((device)) void a(float *address, float b) {
  __ockl_atomic_add_noret_f32(address, b);
}

[Leonc]

• Fix existing test failures.

[bcahoon]

llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp

4415 ↗

(On Diff #447180)

At this location, ILLEGAL is a memory opcode, but SelectionDAGDumper will crash because when trying to print its memops.

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

6636

I think it may be easiest to create the ILLEGAL instruction here rather than creating it after the call to lowerImage. That way, there doesn't have to be copies of the code that creates ILLEGAL.

For example,
MachineSDNode *NewNode = DAG.getMachineNode(AMDGPU::V_ILLEGAL, DL, MVT::Other, Op.getOperand(0));
return DAG.getMergeValues({ DAG.getUNDEF(Op.getValueType()), SDValue(NewNode, 0) }, DL);

The UNDEF value replaces return result for the intrinsic, and the V_ILLEGAL replaces the chain result. It appears, though, that V_ILLEGAL is removed if the intrinsic chain result is not used anywhere. I'm not sure if that is a bad thing since it means it's not really needed?

7121

Can this go in lowerImage? It seems like this will mark some cases as illegal that should be handled either in DAGToDAG or with patterns.

[Leonc]

• Revert changes to return values.
• Refactor code.
• Address comments.

[bcahoon]

llvm/lib/Target/AMDGPU/AMDGPUISelLowering.h

540 ↗

(On Diff #448074)

Will ILLEGAL be removed?

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

6637

No need for braces

8401

The v_illegal should have an operand for the chain operand in the original instruction, if one exists.

t0 EntryToken
res,ch = intrinsic t0, <other opernads>
->
res = undef
ch = v_illegal t0

Also, if there is a use of the chain definition in the original instruction, then there is no need to add the code below.

[arsenm]

Missing testscases. Also should include some assembler and disassembler tests for V_ILLEGAL

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

8401

I'd expect to use the original chain, but it also doesn't really matter given that it's filler anyway

[Leonc]

• Address comments.

[Leonc]

llvm/lib/Target/AMDGPU/AMDGPUISelLowering.h

540 ↗

(On Diff #448074)

Thanks I missed that one.

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

7852

I think this is unnecessary. We still need to call lowerImage for the return value when it succeeds.

8401

Thanks @bcahoon & @arsenm.

I added a check for MemSDNode and pass the chain to V_ILLEGAL if it exists.

[Leonc]

• Handling for atomics plus tests.

[bcahoon]

A couple more tests are needed. A test for for the image intrinsic. Also an assembler test for the v_illegal encoding.

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

7827

no braces needed

llvm/lib/Target/AMDGPU/SIInstructions.td

3368

Do we need different version for gfx10?

[Leonc]

• Add different versions of V_ILLEGAL for different targets.

[foad]

In D123693#3581616, @bcahoon wrote:

The rationale is for library code that looks like:

void example(float *p, float v) {
  if (_ISA_verison == 9008 || __ISA_verison == 9010)
     global_atomic_fadd(p, v);
  else
     generic_atomic_fadd(p, v);
}

At optimization levels > 0, the dead code is eliminated, but at optimization level 0, the dead code remain and the compiler attempts to generate an instruction. That instruction would never be executed due to the condition, but it still needs to be generated. In these cases, we don't have multilibs (target specific libraries).

Right, but why can't global_atomic_fadd and generic_atomic_fadd be functions that just contain a call to the corresponding builtin and have an appropriate "target-cpu"= attribute? I guess you would need to do inlining in the backend once you know what subtarget you are compiling for. Is that viable?

llvm/test/CodeGen/AMDGPU/v_illegal-atomics.ll

2

File should not be executable.

[bcahoon]

llvm/lib/Target/AMDGPU/SIInstructions.td

3364

Do we want different mnemonics, or should the same mnemonic be used for the cases when the encoding is all 0's or all 1's?

When you add the llvm-mc tests, it will also be a good idea to add a test that pipes the output of llvm-mc through llvm-objdump to the that the different encodings show up properly for the two cases.

[Leonc]

• Fix indentation, braces, and file mode.

[Leonc]

llvm/lib/Target/AMDGPU/SIInstructions.td

3364

Do we want different mnemonics, or should the same mnemonic be used for the cases when the encoding is all 0's or all 1's?

@arsenm asked for separate definitions:

I would prefer to define a separate V_ILLEGAL that uses all 1s pre gfx10

[Leonc]

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

8399

Ternary args are the wrong way round. Running local tests before I update the diff to fix this.

[Leonc]

• Fix ternary args.

[Leonc]

• Add v_illegal assembly tests.
• Update instruction definitions.

[Leonc]

llvm/test/MC/AMDGPU/v_illegal-atomics.s

2–3

Adding modifications to pipe the compiler output through objdump as these tests aren't very helpful.

[arsenm]

In D123693#3690864, @foad wrote:

Right, but why can't global_atomic_fadd and generic_atomic_fadd be functions that just contain a call to the corresponding builtin and have an appropriate "target-cpu"= attribute? I guess you would need to do inlining in the backend once you know what subtarget you are compiling for. Is that viable?

The problem is in the called function with the appropriate attributes. The function isn't deleted and we still need to produce something. The current situation is it works in cases where the wrong subtarget also happens to have the same encoding, but breaks on targets that never had an equivalent instruction encoded

[arsenm]

llvm/lib/Target/AMDGPU/SIInstructions.td

3364

The instruction is actually named v_illegal, so you shouldn't invent new names here. I would name these V_ILLEGAL for the gfx10 version and V_ILLEGAL_gfx6_gfx7_gf8_gfx9?

I also think V_ILLEGAL is available in a vop3 encoding (for gfx10) but probably should define that in a separate patch for the benefit of the disassembler

[arsenm]

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

8401

Checking for MemSDNode doesn't make sense, other nodes can have chains. You would need to pass it in from the source if it applies. However I don't think it's worth the complexity, since this really is filler content anyway

[Leonc]

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

8401

Thanks, I agree.

[foad]

In D123693#3696457, @arsenm wrote:

In D123693#3690864, @foad wrote:

Right, but why can't global_atomic_fadd and generic_atomic_fadd be functions that just contain a call to the corresponding builtin and have an appropriate "target-cpu"= attribute? I guess you would need to do inlining in the backend once you know what subtarget you are compiling for. Is that viable?

The problem is in the called function with the appropriate attributes. The function isn't deleted and we still need to produce something. The current situation is it works in cases where the wrong subtarget also happens to have the same encoding, but breaks on targets that never had an equivalent instruction encoded

If target-cpu doesn't let you target different CPUs for different functions then I don't understand what it does.

[arsenm]

In D123693#3696493, @foad wrote:

If target-cpu doesn't let you target different CPUs for different functions then I don't understand what it does.

For the relevant cases in the library, we key off the subtarget feature in target-features. The problem is in the final codegen we've propagated the real cpu to the leaf function, which then artificially has an incompatible target-features. We would have to have some code fixup the target-cpu to some random other device with compatible features

[Leonc]

• Address comments.

[Leonc]

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

8401

It turns out the chain is necessary. An assertion in RemoveNodeFromCSEMaps fails when Op is an atomic intrinsic.

[arsenm]

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

8401

Yes, a chain is needed but you should be able to use the entry node. If you want to forward it in from the call sites, that would also be OK

[Leonc]

• Address comments and fix tests.

[arsenm]

Could also use a disassembler test

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

8397

SDLoc DL(Op)

[Leonc]

• Address comments.

[arsenm]

llvm/test/CodeGen/AMDGPU/v_illegal-atomics.ll

41

This looks backwards. gfx10 is getting the -1 encoding and gfx9 is getting 0

54

gfx10 has v_illegal, so this should be 0x0 and printed as v_illegal

[Leonc]

llvm/test/CodeGen/AMDGPU/v_illegal-atomics.ll

41

Are you sure gfx9 is getting the zero encoding? I thought this line was "address: encoding".
If this was wrong wouldn't the tests in v_illegal-atomics.s also be wrong?

[arsenm]

llvm/test/CodeGen/AMDGPU/v_illegal-atomics.ll

41

You're right, gfx9 looks right. The gfx1030 line with .long 0xffffffff looks wrong.

[Leonc]

llvm/test/CodeGen/AMDGPU/v_illegal-atomics.ll

41

I'm guessing gfx1030 is due to the disassebler. We know the encoding is correct from the tests in v_illegal-atomics.s.
I'll look into it.

[Leonc]

• Address comments.

[arsenm]

LGTM except for the stray debug print. We also probably should clean up the the encoding to rely on subtarget features instead of the generation check

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

8396

DRop this debug printing

[Leonc]

• Remove debug code.

[Leonc]

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

8396

Thanks, my mistake.

[Leonc]

In D123693#3703089, @arsenm wrote:

We also probably should clean up the the encoding to rely on subtarget features instead of the generation check

Is there a generic way to do that without adding maintenance overhead, or is it better that we don't implicitly support new subtargets/generations in case they have different encodings of v_illegal?

llvm/lib/Target/AMDGPU/SIISelLowering.h

private:
 
   SDValue lowerWorkitemID(SelectionDAG &DAG, SDValue Op, unsigned Dim,
                           const ArgDescriptor &ArgDesc) const;
 
   SDValue LowerINTRINSIC_WO_CHAIN(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerINTRINSIC_W_CHAIN(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerINTRINSIC_VOID(SDValue Op, SelectionDAG &DAG) const;
 
+  SDValue makeV_ILLEGAL(SDValue Op, SelectionDAG &DAG) const;
+
   // The raw.tbuffer and struct.tbuffer intrinsics have two offset args: offset
   // (the offset that is included in bounds checking and swizzling, to be split
   // between the instruction's voffset and immoffset fields) and soffset (the
   // offset that is excluded from bounds checking and swizzling, to go in the
   // instruction's soffset field).  This function takes the first kind of
   // offset and figures out how to split it between voffset and immoffset.
   std::pair<SDValue, SDValue> splitBufferOffsets(SDValue Offset,
                                                  SelectionDAG &DAG) const;

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

Opcode = AMDGPU::getMIMGOpcode(IntrOpcode,
                                    UseNSA ? AMDGPU::MIMGEncGfx10NSA
                                           : AMDGPU::MIMGEncGfx10Default,
                                    NumVDataDwords, NumVAddrDwords);
   } else {
     if (Subtarget->hasGFX90AInsts()) {
       Opcode = AMDGPU::getMIMGOpcode(IntrOpcode, AMDGPU::MIMGEncGfx90a,
                                      NumVDataDwords, NumVAddrDwords);
       if (Opcode == -1)
-        report_fatal_error(
-            "requested image instruction is not supported on this GPU");
+        return makeV_ILLEGAL(Op, DAG);
     }
     if (Opcode == -1 &&
         Subtarget->getGeneration() >= AMDGPUSubtarget::VOLCANIC_ISLANDS)
       Opcode = AMDGPU::getMIMGOpcode(IntrOpcode, AMDGPU::MIMGEncGfx8,
                                      NumVDataDwords, NumVAddrDwords);
     if (Opcode == -1)
       Opcode = AMDGPU::getMIMGOpcode(IntrOpcode, AMDGPU::MIMGEncGfx6,
                                      NumVDataDwords, NumVAddrDwords);
   }
   assert(Opcode != -1);
static void updateBufferMMO(MachineMemOperand *MMO, SDValue VOffset,
                             SDValue VIndex = SDValue()) {
   if (!isa<ConstantSDNode>(VOffset) || !isa<ConstantSDNode>(SOffset) ||
       !isa<ConstantSDNode>(Offset)) {
     // The combined offset is not known to be constant, so we cannot represent
     // it in the MMO. Give up.
     MMO->setValue((Value *)nullptr);
     return;
   }
 
   if (VIndex && (!isa<ConstantSDNode>(VIndex) ||
                  !cast<ConstantSDNode>(VIndex)->isZero())) {
     // The strided index component of the address is not known to be zero, so we
     // cannot represent it in the MMO. Give up.
     MMO->setValue((Value *)nullptr);
     return;
   }
 
case Intrinsic::amdgcn_flat_atomic_fmax: {
     SDValue Ops[] = {
       M->getOperand(0), // Chain
       M->getOperand(2), // Ptr
       M->getOperand(3)  // Value
     };
     unsigned Opcode = 0;
     switch (IntrID) {
     case Intrinsic::amdgcn_global_atomic_fadd:
+      if (!Subtarget->hasAtomicFaddNoRtnInsts())
+        return makeV_ILLEGAL(Op, DAG);
+      LLVM_FALLTHROUGH;
     case Intrinsic::amdgcn_flat_atomic_fadd: {
       EVT VT = Op.getOperand(3).getValueType();
       return DAG.getAtomic(ISD::ATOMIC_LOAD_FADD, DL, VT,
                            DAG.getVTList(VT, MVT::Other), Ops,
                            M->getMemOperand());
     }
     case Intrinsic::amdgcn_global_atomic_fmin:
     case Intrinsic::amdgcn_flat_atomic_fmin: {
       Opcode = AMDGPUISD::ATOMIC_LOAD_FMIN;
       break;
     }
     case Intrinsic::amdgcn_global_atomic_fmax:
     case Intrinsic::amdgcn_flat_atomic_fmax: {
       Opcode = AMDGPUISD::ATOMIC_LOAD_FMAX;
       break;
     }
     default:
       llvm_unreachable("unhandled atomic opcode");
     }
     return DAG.getMemIntrinsicNode(Opcode, SDLoc(Op),
                                    M->getVTList(), Ops, M->getMemoryVT(),
                                    M->getMemOperand());
   }
   default:
 
     if (const AMDGPU::ImageDimIntrinsicInfo *ImageDimIntr =
             AMDGPU::getImageDimIntrinsicInfo(IntrID))
       return lowerImage(Op, ImageDimIntr, DAG, true);
 
     return SDValue();
   }
 }
 
 // Call DAG.getMemIntrinsicNode for a load, but first widen a dwordx3 type to
 // dwordx4 if on SI.
 SDValue SITargetLowering::getMemIntrinsicNode(unsigned Opcode, const SDLoc &DL,
                                               SDVTList VTList,
                                               ArrayRef<SDValue> Ops, EVT MemVT,
                                               MachineMemOperand *MMO,
                                               SelectionDAG &DAG) const {
   EVT VT = VTList.VTs[0];
   EVT WidenedVT = VT;
   EVT WidenedMemVT = MemVT;
   if (!Subtarget->hasDwordx3LoadStores() &&
if (const AMDGPU::ImageDimIntrinsicInfo *ImageDimIntr =
             AMDGPU::getImageDimIntrinsicInfo(IntrinsicID))
       return lowerImage(Op, ImageDimIntr, DAG, true);
 
     return Op;
   }
   }
 }
 
+SDValue SITargetLowering::makeV_ILLEGAL(SDValue Op, SelectionDAG & DAG) const {
+  // Create the V_ILLEGAL node.
+  SDLoc DL(Op);
+  auto Opcode = Subtarget->getGeneration() < AMDGPUSubtarget::GFX10 ?
+    AMDGPU::V_ILLEGAL_gfx6_gfx7_gfx8_gfx9 : AMDGPU::V_ILLEGAL;
+  auto EntryNode = DAG.getEntryNode();
+  auto IllegalNode = DAG.getMachineNode(Opcode, DL, MVT::Other, EntryNode);
+  auto IllegalVal = SDValue(IllegalNode, 0u);
+
+  // Add the V_ILLEGAL node to the root chain to prevent its removal.
+  auto Chains = SmallVector<SDValue, 2u>();
+  Chains.push_back(IllegalVal);
+  Chains.push_back(DAG.getRoot());
+  auto Root = DAG.getTokenFactor(SDLoc(Chains.back()), Chains);
+  DAG.setRoot(Root);
+
+  // Merge with UNDEF to satisfy return value requirements.
+  auto UndefVal = DAG.getUNDEF(Op.getValueType());
+  return DAG.getMergeValues({UndefVal, IllegalVal}, DL);
+}
+
 // The raw.(t)buffer and struct.(t)buffer intrinsics have two offset args:
 // offset (the offset that is included in bounds checking and swizzling, to be
 // split between the instruction's voffset and immoffset fields) and soffset
 // (the offset that is excluded from bounds checking and swizzling, to go in
 // the instruction's soffset field).  This function takes the first kind of
 // offset and figures out how to split it between voffset and immoffset.
 std::pair<SDValue, SDValue> SITargetLowering::splitBufferOffsets(
     SDValue Offset, SelectionDAG &DAG) const {

llvm/lib/Target/AMDGPU/SIInstructions.td

def G_FPTRUNC_ROUND_UPWARD : AMDGPUGenericInstruction {
   let hasSideEffects = 0;
 }
 
 def G_FPTRUNC_ROUND_DOWNWARD : AMDGPUGenericInstruction {
   let OutOperandList = (outs type0:$vdst);
   let InOperandList = (ins type1:$src0);
   let hasSideEffects = 0;
 }
+
+//============================================================================//
+// Dummy Instructions
+//============================================================================//
+
+def V_ILLEGAL_gfx6_gfx7_gfx8_gfx9 : Enc32, InstSI<(outs), (ins), "v_illegal"> {
+  let Inst{31-0} = 0xFFFFFFFF;
+  let FixedSize = 1;
+  let Size = 4;
+  let Uses = [EXEC];
+  let hasSideEffects = 1;
+  let SubtargetPredicate = isGFX6GFX7GFX8GFX9;
+}
+
+def V_ILLEGAL : Enc32, InstSI<(outs), (ins), "v_illegal"> {
+  let Inst{31-0} = 0x00000000;
+  let FixedSize = 1;
+  let Size = 4;
+  let Uses = [EXEC];
+  let hasSideEffects = 1;
+  let SubtargetPredicate = isGFX10Plus;
+}

llvm/test/CodeGen/AMDGPU/v_illegal-atomics.ll

+; RUN: llc -march=amdgcn -mcpu=gfx906 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX906-ASM %s
+; RUN: llc -march=amdgcn -mcpu=gfx908 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX908-ASM %s
+; RUN: llc -march=amdgcn -mcpu=gfx90a -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX90A-ASM %s
+; RUN: llc -march=amdgcn -mcpu=gfx940 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX940-ASM %s
+; RUN: llc -march=amdgcn -mcpu=gfx1030 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX1030-ASM %s
+; RUN: llc -march=amdgcn -mcpu=gfx1100 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX1100-ASM %s
+
+; GFX906-ASM-LABEL: fadd_test:
+; GFX906-ASM-NOT:   global_atomic_add_f32
+; GFX906-ASM:       v_illegal
+
+; GFX908-ASM-LABEL: fadd_test:
+; GFX908-ASM-NOT:   v_illegal
+; GFX908-ASM:       global_atomic_add_f32
+
+; GFX90A-ASM-LABEL: fadd_test:
+; GFX90A-ASM-NOT:   v_illegal
+; GFX90A-ASM:       global_atomic_add_f32
+
+; GFX940-ASM-LABEL: fadd_test:
+; GFX940-ASM-NOT:   v_illegal
+; GFX940-ASM:       global_atomic_add_f32
+
+; GFX1030-ASM-LABEL: fadd_test:
+; GFX1030-ASM-NOT:   global_atomic_add_f32
+; GFX1030-ASM:       v_illegal
+
+; GFX1100-ASM-LABEL: fadd_test:
+; GFX1100-ASM-NOT:   v_illegal
+; GFX1100-ASM:       global_atomic_add_f32
+
+; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx906 -filetype=obj -verify-machineinstrs < %s | llvm-objdump --triple=amdgcn--amdhsa --mcpu=gfx906 -d - | FileCheck --check-prefix=GFX906-OBJ %s
+; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx908 -filetype=obj -verify-machineinstrs < %s | llvm-objdump --triple=amdgcn--amdhsa --mcpu=gfx908 -d - | FileCheck --check-prefix=GFX908-OBJ %s
+; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx90a -filetype=obj -verify-machineinstrs < %s | llvm-objdump --triple=amdgcn--amdhsa --mcpu=gfx90a -d - | FileCheck --check-prefix=GFX90A-OBJ %s
+; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx940 -filetype=obj -verify-machineinstrs < %s | llvm-objdump --triple=amdgcn--amdhsa --mcpu=gfx940 -d - | FileCheck --check-prefix=GFX940-OBJ %s
+; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx1030 -filetype=obj -verify-machineinstrs < %s | llvm-objdump --triple=amdgcn--amdhsa --mcpu=gfx1030 -d - | FileCheck --check-prefix=GFX1030-OBJ %s
+; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx1100 -filetype=obj -verify-machineinstrs < %s | llvm-objdump --triple=amdgcn--amdhsa --mcpu=gfx1100 -d - | FileCheck --check-prefix=GFX1100-OBJ %s
+
+; GFX906-OBJ:      s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX906-OBJ-NEXT: v_illegal // 000000000004: FFFFFFFF
+
+; GFX908-OBJ:      s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX908-OBJ-NEXT: global_atomic_add_f32
+
+; GFX90A-OBJ:      s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX90A-OBJ-NEXT: global_atomic_add_f32
+
+; GFX940-OBJ:      s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX940-OBJ-NEXT: global_atomic_add_f32
+
+; GFX1030-OBJ:      s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX1030-OBJ-NEXT: s_waitcnt_vscnt null, 0x0
+; GFX1030-OBJ-NEXT: v_illegal // 000000000008: 00000000
+
+; GFX1100-OBJ:      s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX1100-OBJ-NEXT: s_waitcnt_vscnt null, 0x0
+; GFX1100-OBJ-NEXT: global_atomic_add_f32 v[0:1], v2, off
+
+define fastcc void @fadd_test(float addrspace(1)* nocapture noundef %0, float noundef %1) unnamed_addr {
+  %3 = tail call float @llvm.amdgcn.global.atomic.fadd.f32.p1f32.f32(float addrspace(1)* noundef %0, float noundef %1)
+  ret void
+}
+declare float @llvm.amdgcn.global.atomic.fadd.f32.p1f32.f32(float addrspace(1)* nocapture, float)

llvm/test/CodeGen/AMDGPU/v_illegal-image_sample.ll

+; RUN: llc -O0 -march=amdgcn -mcpu=gfx906 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX906 %s
+; RUN: llc -O0 -march=amdgcn -mcpu=gfx908 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX908 %s
+; RUN: llc -O0 -march=amdgcn -mcpu=gfx90a -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX90A %s
+; RUN: llc -O0 -march=amdgcn -mcpu=gfx940 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX940 %s
+; RUN: llc -O0 -march=amdgcn -mcpu=gfx1030 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX1030 %s
+; RUN: llc -O0 -march=amdgcn -mcpu=gfx1100 -verify-machineinstrs < %s | FileCheck -check-prefixes=GFX1100 %s
+
+; GFX906-LABEL: image_sample_test:
+; GFX906-NOT: v_illegal
+; GFX906: image_sample_lz
+
+; GFX908-LABEL: image_sample_test:
+; GFX908-NOT: v_illegal
+; GFX908: image_sample_lz
+
+; GFX90A-LABEL: image_sample_test:
+; GFX90A-NOT: image_sample_lz
+; GFX90A: v_illegal
+
+; GFX940-LABEL: image_sample_test:
+; GFX940-NOT: image_sample_lz
+; GFX940: v_illegal
+
+; GFX1030-LABEL: image_sample_test:
+; GFX1030-NOT: v_illegal
+; GFX1030: image_sample_lz
+
+; GFX1100-LABEL: image_sample_test:
+; GFX1100-NOT: v_illegal
+; GFX1100: image_sample_lz
+
+define amdgpu_kernel void @image_sample_test(<4 x float> addrspace(1)* %out, float %arg1, float %arg2, <8 x i32> %arg3, <4 x i32> %arg4) {
+  
+  %result = tail call <4 x float> @llvm.amdgcn.image.sample.lz.2d.v4f32.f32(i32 15, float %arg1, float %arg2, <8 x i32> %arg3, <4 x i32> %arg4, i1 false, i32 0, i32 0)
+
+  store <4 x float> %result, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+declare <4 x float> @llvm.amdgcn.image.sample.lz.2d.v4f32.f32(i32 immarg, float, float, <8 x i32>, <4 x i32>, i1 immarg, i32 immarg, i32 immarg)

llvm/test/MC/AMDGPU/v_illegal-atomics.s

+// RUN: llvm-mc -arch=amdgcn -mcpu=gfx906 -show-encoding %s | FileCheck --check-prefix=GFX906 %s
+// RUN: llvm-mc -arch=amdgcn -mcpu=gfx908 -show-encoding %s | FileCheck --check-prefix=GFX908 %s
+// RUN: llvm-mc -arch=amdgcn -mcpu=gfx90a -show-encoding %s | FileCheck --check-prefix=GFX90A %s
+// RUN: llvm-mc -arch=amdgcn -mcpu=gfx940 -show-encoding %s | FileCheck --check-prefix=GFX940 %s
+// RUN: llvm-mc -arch=amdgcn -mcpu=gfx1030 -show-encoding %s | FileCheck --check-prefix=GFX1030 %s
+// RUN: llvm-mc -arch=amdgcn -mcpu=gfx1100 -show-encoding %s | FileCheck --check-prefix=GFX1100 %s
+
+v_illegal
+// GFX906: encoding: [0xff,0xff,0xff,0xff]
+// GFX908: encoding: [0xff,0xff,0xff,0xff]
+// GFX90A: encoding: [0xff,0xff,0xff,0xff]
+// GFX940: encoding: [0xff,0xff,0xff,0xff]
+// GFX1030: encoding: [0x00,0x00,0x00,0x00]
+// GFX1100: encoding: [0x00,0x00,0x00,0x00]