AMDGPU/SI: Implement amdgcn image intrinsics

[• tstellarAMD]

Summary

This patch implements amdgcn image intrinsics. The flags are passed as a mask parameter.

[arsenm]

include/llvm/IR/IntrinsicsAMDGPU.td

203

The full instruction name is image_get_resinfo, so the intrinsic should be int_amdgcn_image_getresinfo

211

This requires a descriptive comment (including the values for which bits)

276–307

These are also missing the image part of the name as well

lib/Target/AMDGPU/AMDGPUISelDAGToDAG.cpp

156–162 ↗

(On Diff #65635)

Param names should be capitalized, the function name itself should be lowercase

1103–1116 ↗

(On Diff #65635)

You don't need these intermediate variables, you can just check the bit as the argument to getTargetConstant directly

1134–1135 ↗

(On Diff #65635)

Commented out code

lib/Target/AMDGPU/SIDefines.h

111–114 ↗

(On Diff #65635)

Not all of these bits should be exposed. GLC, SLC, and TFE definitely should not be. TFE changes the register class of the output. This should strictly be the ones for controlling the sampling

lib/Target/AMDGPU/SIInstructions.td

2738

Not sure what this means

2740–2749

I think you can reduce the number of repeated lines by passing in the suffix, and then concat + cast to the intrinsic (Similar to how MUBUF_LoadIntrinsicPat does it)

test/CodeGen/AMDGPU/llvm.amdgcn.gather4.ll

4–5 ↗

(On Diff #65635)

Space between ; and start of comment. Also should probably switch to GCN prefix because it seems likely we'll need to distinguish subtargets here

9–13 ↗

(On Diff #65635)

It should work to just store the output directly into an output pointer rather than having to scalarize and export since this is a compute shader

[• tstellarAMD]

include/llvm/IR/IntrinsicsAMDGPU.td

198–201

Changing this intrinsic will break Mesa, we will need to update Mesa before we can commit this.

[nhaehnle]

include/llvm/IR/IntrinsicsAMDGPU.td

193–198

Can we just not do this kind of change, please? I don't see how it improves anything, it's inconsistent with the ISA description which has the flags separate, and it'll require an annoying flag day synchronization with Mesa.

[nhaustov]

include/llvm/IR/IntrinsicsAMDGPU.td

193–198

I agree. Separate flags also play nicely with assembler.

[cfang]

include/llvm/IR/IntrinsicsAMDGPU.td

193–198

We plan to add and expose d16 bit, so the Intrinsics for Mesa has to be updated anyway. For the future chips, we may add more flag (bit) and all applications (including Mesa) have to be updated every time a new flag is added. One advantage of using mask parameter is that we don't have to update the application if the new bit is not exposed.

198–201

We will have to add d16 bit! So Mesa will have to be update anyway.

[arsenm]

include/llvm/IR/IntrinsicsAMDGPU.td

193–198

The assembler has nothing to do with the intrinsic definition. This isn't changing the MachineInstr's operand structure

[cfang]

update the patch based on Matt's comments.

The following comments are not updated:

1. >Not all of these bits should be exposed. GLC, SLC, and TFE definitely should not be. TFE changes the register class of the output. This should strictly be the ones for controlling the sampling
   • I asked this question before I started, and the agreement is to expose all at this moment. We have to study all these flags in detail and get a short list to expose later.

2. >I think you can reduce the number of repeated lines by passing in the suffix, and then concat + cast to the intrinsic (Similar to how MUBUF_LoadIntrinsicPat does it)
   • what's the problem of repeated lines? I think it is more readable than simplified one. Or we can simplify the def in the TableGen cleanup work.

NOTE that Mesa has to be update to accommodate the image_load/image_store intrinsic parameter change.

[cfang]

Remove the RSVD bit based on Tom's suggestion.

[mareko]

BTW, SLC and GLC bits definitely SHOULD be exposed. LLVM isn't high-level enough to be able to make any assumptions about cache strategies.

[arsenm]

In D22838#501936, @mareko wrote:

BTW, SLC and GLC bits definitely SHOULD be exposed. LLVM isn't high-level enough to be able to make any assumptions about cache strategies.

GLC and SLC should still be separate parameters from the format mask (or metadata should control these like it will for non temporal regular loads/stores)

[• tstellarAMD]

This comment has been deleted.

include/llvm/IR/IntrinsicsAMDGPU.td

193–198

Can we just not do this kind of change, please? I don't see how it improves anything, it's inconsistent with the ISA description which has the flags separate, and it'll require an annoying flag day synchronization with Mesa.

I really prefer using a mask over having to create an entire new set of intrinsics each time we have to add a new bit. I think we have two solutions here:

1. As Matt has suggested, keep mesa the same, and have the auto-upgrader in LLVM change from the old intrinsics to the new mask style intrinsics. Although, with this solution, I think we'd still eventually want/need Mesa to start using the mask version.

2. Define the intrinsics as var_arg. This would allow us to add to i1 operands without breaking the existing operands. I'm just not sure how well var_arg intrinsics are supported.

[• tstellarAMD]

include/llvm/IR/IntrinsicsAMDGPU.td

221–228

Moving the sample intrinsics to this file is unrelated to the AMDGPUImageLoad/AMDGPUImageStore changes, so this should be done in a separate patch.

[cfang]

include/llvm/IR/IntrinsicsAMDGPU.td

221–228

The patch is to implement amdgcn image inttrinsics, which has three categories: AMDGPUImageLoad, AMDGPUImageStore and AMDGPUImageSample. While AMDGPUImageSample is newly defined, and the other two are modified, they do use the same mechanism, i.e. mask parameter! I think it should be better for them to be together in one patch.

[cfang]

Do not use mask parameter!

Keep image load/store intrinsics

expose all flags for sampler intrinsics

[• tstellarAMD]

include/llvm/IR/IntrinsicsAMDGPU.td

206

This should go in a separate patch. This patch should be only the sampler changes.

218

This is an unrelated change.

224

I'm thinking vdata should be llvm_anyfloat_ty, so we can have it return <4 x half> for the d16 operations. Though it's going to be weird that some <4 x half> values take 4 registers and some only take two.

Another thing I'm not sure of is if image samplers always return floating-point values and never integers.

226

This should be changed to llvm_anyint_ty, so that we can infer the r128 bit.

232

This r128 bit should be dropped.

lib/Target/AMDGPU/SIInstructions.td

2722–2729

These image load/store changes should go in a separate patch.

[nhaehnle]

Thanks for doing this. One comment below.

include/llvm/IR/IntrinsicsAMDGPU.td

233

tfe should be dropped. AFAIU it changes the return type (5 return values instead of 4).

[cfang]

Implement sampler intrinsics:

1. define vdata type to be llvm_anyfloat_ty, address type to be llvm_anyfloat_ty, and rsrc type to be llvm_anyint_ty as a result, we expect the intrinsics name to have three suffixes to overload each of these three types;

2. D128 as well as two other flags are implied in the three types, for example, if you use v8i32 as rsrc type, then r128 is true!

3. don't expose TFE flag, and other flags are exposed in the instruction order: unrm, glc, slc, lwe and da.

4. LIT tests are not included now!

[• tstellarAMD]

The intrinsic definitions look good now, thanks.

[cfang]

Add LIT tests for sampler intrinsics!

[• tstellarAMD]

Re-base on top of trunk.

[arsenm]

LGTM

include/llvm/IR/IntrinsicsAMDGPU.td

    llvm_v8i32_ty,     // rsrc(SGPR)
    llvm_i32_ty,       // dmask(imm)
    llvm_i1_ty,        // r128(imm)
    llvm_i1_ty,        // da(imm)
    llvm_i1_ty,        // glc(imm)
    llvm_i1_ty],       // slc(imm)
   [IntrReadMem]>;
 
 def int_amdgcn_image_load : AMDGPUImageLoad;
 def int_amdgcn_image_load_mip : AMDGPUImageLoad;
+def int_amdgcn_image_getresinfo : AMDGPUImageLoad;
 
 class AMDGPUImageStore : Intrinsic <
   [],
   [llvm_v4f32_ty,     // vdata(VGPR)
    llvm_anyint_ty,    // vaddr(VGPR)
    llvm_v8i32_ty,     // rsrc(SGPR)
    llvm_i32_ty,       // dmask(imm)
    llvm_i1_ty,        // r128(imm)
    llvm_i1_ty,        // da(imm)
    llvm_i1_ty,        // glc(imm)
    llvm_i1_ty],       // slc(imm)
-  []>;
+  [IntrWriteMem]>;
 
 def int_amdgcn_image_store : AMDGPUImageStore;
 def int_amdgcn_image_store_mip : AMDGPUImageStore;
 
+class AMDGPUImageSample : Intrinsic <
+    [llvm_v4f32_ty], // vdata(VGPR)
+    [llvm_anyfloat_ty,  // vaddr(VGPR)
+     llvm_v8i32_ty,     // rsrc(SGPR)
+     llvm_v4i32_ty,      // sampler(SGPR)
+     llvm_i32_ty,        // dmask(imm)
+     llvm_i1_ty,         // unorm(imm)
+     llvm_i1_ty,         // glc(imm)
+     llvm_i1_ty,         // slc(imm)
+     llvm_i1_ty,         // r128(imm)
+     llvm_i1_ty,         // tfe(imm)
+     llvm_i1_ty,         // lwe(imm)
+     llvm_i1_ty],        // da(imm)
+     [IntrReadMem]>;
+
+// Basic sample
+def int_amdgcn_image_sample : AMDGPUImageSample;
+def int_amdgcn_image_sample_cl : AMDGPUImageSample;
+def int_amdgcn_image_sample_d : AMDGPUImageSample;
+def int_amdgcn_image_sample_d_cl : AMDGPUImageSample;
+def int_amdgcn_image_sample_l : AMDGPUImageSample;
+def int_amdgcn_image_sample_b : AMDGPUImageSample;
+def int_amdgcn_image_sample_b_cl : AMDGPUImageSample;
+def int_amdgcn_image_sample_lz : AMDGPUImageSample;
+def int_amdgcn_image_sample_cd : AMDGPUImageSample;
+def int_amdgcn_image_sample_cd_cl : AMDGPUImageSample;
+
+// Sample with comparison
+def int_amdgcn_image_sample_c : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_cl : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_d : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_d_cl : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_l : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_b : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_b_cl : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_lz : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_cd : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_cd_cl : AMDGPUImageSample;
+
+// Sample with offsets
+def int_amdgcn_image_sample_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_cl_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_d_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_d_cl_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_l_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_b_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_b_cl_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_lz_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_cd_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_cd_cl_o : AMDGPUImageSample;
+
+// Sample with comparison and offsets
+def int_amdgcn_image_sample_c_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_cl_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_d_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_d_cl_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_l_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_b_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_b_cl_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_lz_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_cd_o : AMDGPUImageSample;
+def int_amdgcn_image_sample_c_cd_cl_o : AMDGPUImageSample;
+
+// Basic gather4
+def int_amdgcn_image_gather4 : AMDGPUImageSample;
+def int_amdgcn_image_gather4_cl : AMDGPUImageSample;
+def int_amdgcn_image_gather4_l : AMDGPUImageSample;
+def int_amdgcn_image_gather4_b : AMDGPUImageSample;
+def int_amdgcn_image_gather4_b_cl : AMDGPUImageSample;
+def int_amdgcn_image_gather4_lz : AMDGPUImageSample;
+
+// Gather4 with comparison
+def int_amdgcn_image_gather4_c : AMDGPUImageSample;
+def int_amdgcn_image_gather4_c_cl : AMDGPUImageSample;
+def int_amdgcn_image_gather4_c_l : AMDGPUImageSample;
+def int_amdgcn_image_gather4_c_b : AMDGPUImageSample;
+def int_amdgcn_image_gather4_c_b_cl : AMDGPUImageSample;
+def int_amdgcn_image_gather4_c_lz : AMDGPUImageSample;
+
+// Gather4 with offsets
+def int_amdgcn_image_gather4_o : AMDGPUImageSample;
+def int_amdgcn_image_gather4_cl_o : AMDGPUImageSample;
+def int_amdgcn_image_gather4_l_o : AMDGPUImageSample;
+def int_amdgcn_image_gather4_b_o : AMDGPUImageSample;
+def int_amdgcn_image_gather4_b_cl_o : AMDGPUImageSample;
+def int_amdgcn_image_gather4_lz_o : AMDGPUImageSample;
+
+// Gather4 with comparison and offsets
+def int_amdgcn_image_gather4_c_o : AMDGPUImageSample;
+def int_amdgcn_image_gather4_c_cl_o : AMDGPUImageSample;
+def int_amdgcn_image_gather4_c_l_o : AMDGPUImageSample;
+def int_amdgcn_image_gather4_c_b_o : AMDGPUImageSample;
+def int_amdgcn_image_gather4_c_b_cl_o : AMDGPUImageSample;
+def int_amdgcn_image_gather4_c_lz_o : AMDGPUImageSample;
+
+def int_amdgcn_image_getlod : AMDGPUImageSample;
+
+
+
 class AMDGPUImageAtomic : Intrinsic <
   [llvm_i32_ty],
   [llvm_i32_ty,       // vdata(VGPR)
    llvm_anyint_ty,    // vaddr(VGPR)
    llvm_v8i32_ty,     // rsrc(SGPR)
    llvm_i1_ty,        // r128(imm)
    llvm_i1_ty,        // da(imm)
    llvm_i1_ty],       // slc(imm)
   []>;
 

lib/Target/AMDGPU/SIInstructions.td

 >;
 
 multiclass SampleRawPatterns<SDPatternOperator name, string opcode> {
   def : SampleRawPattern<name, !cast<MIMG>(opcode # _V4_V1), i32>;
   def : SampleRawPattern<name, !cast<MIMG>(opcode # _V4_V2), v2i32>;
   def : SampleRawPattern<name, !cast<MIMG>(opcode # _V4_V4), v4i32>;
   def : SampleRawPattern<name, !cast<MIMG>(opcode # _V4_V8), v8i32>;
   def : SampleRawPattern<name, !cast<MIMG>(opcode # _V4_V16), v16i32>;
 }
 
+
+// Image + sampler for amdgcn
+class AMDGCNSamplePattern<SDPatternOperator name, MIMG opcode, ValueType vt> : Pat <
+  (name vt:$addr, v8i32:$rsrc, v4i32:$sampler, i32:$dmask, i1:$unorm, i1:$glc,
+        i1:$slc, i1:$r128, i1:$tfe, i1:$lwe, i1:$da),
+  (opcode $addr, $rsrc, $sampler,
+          (as_i32imm $dmask), (as_i1imm $unorm), (as_i1imm $glc), (as_i1imm $slc),
+          (as_i1imm $r128), (as_i1imm $tfe), (as_i1imm $lwe), (as_i1imm $da))
+>;
+
+multiclass AMDGCNSamplePatterns<SDPatternOperator name, string opcode> {
+  def : AMDGCNSamplePattern<name, !cast<MIMG>(opcode # _V4_V1), f32>;
+  def : AMDGCNSamplePattern<name, !cast<MIMG>(opcode # _V4_V2), v2f32>;
+  def : AMDGCNSamplePattern<name, !cast<MIMG>(opcode # _V4_V4), v4f32>;
+  def : AMDGCNSamplePattern<name, !cast<MIMG>(opcode # _V4_V8), v8f32>;
+  def : AMDGCNSamplePattern<name, !cast<MIMG>(opcode # _V4_V16), v16f32>;
+}
+
+
 // Image only
 class ImagePattern<SDPatternOperator name, MIMG opcode, ValueType vt> : Pat <
   (name vt:$addr, v8i32:$rsrc, imm:$dmask, imm:$unorm,
         imm:$r128, imm:$da, imm:$glc, imm:$slc, imm:$tfe, imm:$lwe),
   (opcode $addr, $rsrc,
           (as_i32imm $dmask), (as_i1imm $unorm), (as_i1imm $glc), (as_i1imm $slc),
           (as_i1imm $r128), (as_i1imm $tfe), (as_i1imm $lwe), (as_i1imm $da))
 >;
 
 multiclass ImagePatterns<SDPatternOperator name, string opcode> {

 
 class ImageAtomicCmpSwapPattern<MIMG opcode, ValueType vt> : Pat <
   (int_amdgcn_image_atomic_cmpswap i32:$vsrc, i32:$vcmp, vt:$addr, v8i32:$rsrc,
                                    imm:$r128, imm:$da, imm:$slc),
   (EXTRACT_SUBREG
     (opcode (REG_SEQUENCE VReg_64, $vsrc, sub0, $vcmp, sub1),
             $addr, $rsrc, 3, 1, 1, (as_i1imm $slc), (as_i1imm $r128), 0, 0, (as_i1imm $da)),
     sub0)
 >;
 
+// ======= SI Image Intrinsics ================
+
+// Image load
+defm : ImagePatterns<int_SI_image_load, "IMAGE_LOAD">;
+defm : ImagePatterns<int_SI_image_load_mip, "IMAGE_LOAD_MIP">;
+def : ImagePattern<int_SI_getresinfo, IMAGE_GET_RESINFO_V4_V1, i32>;
+
 // Basic sample
 defm : SampleRawPatterns<int_SI_image_sample,           "IMAGE_SAMPLE">;
 defm : SampleRawPatterns<int_SI_image_sample_cl,        "IMAGE_SAMPLE_CL">;
 defm : SampleRawPatterns<int_SI_image_sample_d,         "IMAGE_SAMPLE_D">;
 defm : SampleRawPatterns<int_SI_image_sample_d_cl,      "IMAGE_SAMPLE_D_CL">;
 defm : SampleRawPatterns<int_SI_image_sample_l,         "IMAGE_SAMPLE_L">;
 defm : SampleRawPatterns<int_SI_image_sample_b,         "IMAGE_SAMPLE_B">;
 defm : SampleRawPatterns<int_SI_image_sample_b_cl,      "IMAGE_SAMPLE_B_CL">;
 defm : SampleRawPatterns<int_SI_image_sample_lz,        "IMAGE_SAMPLE_LZ">;
 defm : SampleRawPatterns<int_SI_image_sample_cd,        "IMAGE_SAMPLE_CD">;

 def : SampleRawPattern<int_SI_gather4_c_l_o,     IMAGE_GATHER4_C_L_O_V4_V8,  v8i32>;
 def : SampleRawPattern<int_SI_gather4_c_b_o,     IMAGE_GATHER4_C_B_O_V4_V8,  v8i32>;
 def : SampleRawPattern<int_SI_gather4_c_b_cl_o,  IMAGE_GATHER4_C_B_CL_O_V4_V8, v8i32>;
 def : SampleRawPattern<int_SI_gather4_c_lz_o,    IMAGE_GATHER4_C_LZ_O_V4_V4, v4i32>;
 def : SampleRawPattern<int_SI_gather4_c_lz_o,    IMAGE_GATHER4_C_LZ_O_V4_V8, v8i32>;
 
 def : SampleRawPattern<int_SI_getlod, IMAGE_GET_LOD_V4_V1, i32>;
 def : SampleRawPattern<int_SI_getlod, IMAGE_GET_LOD_V4_V2, v2i32>;
 def : SampleRawPattern<int_SI_getlod, IMAGE_GET_LOD_V4_V4, v4i32>;
 
-def : ImagePattern<int_SI_getresinfo, IMAGE_GET_RESINFO_V4_V1, i32>;
-defm : ImagePatterns<int_SI_image_load, "IMAGE_LOAD">;
-defm : ImagePatterns<int_SI_image_load_mip, "IMAGE_LOAD_MIP">;
+
+// ======= amdgcn Image Intrinsics ==============
+
+// Image load
 defm : ImageLoadPatterns<int_amdgcn_image_load, "IMAGE_LOAD">;
 defm : ImageLoadPatterns<int_amdgcn_image_load_mip, "IMAGE_LOAD_MIP">;
+def : ImageLoadPattern<int_amdgcn_image_getresinfo, IMAGE_GET_RESINFO_V4_V1, i32>;
+
+// Image store
 defm : ImageStorePatterns<int_amdgcn_image_store, "IMAGE_STORE">;
 defm : ImageStorePatterns<int_amdgcn_image_store_mip, "IMAGE_STORE_MIP">;
+
+// Basic sample
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample,           "IMAGE_SAMPLE">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_cl,        "IMAGE_SAMPLE_CL">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_d,         "IMAGE_SAMPLE_D">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_d_cl,      "IMAGE_SAMPLE_D_CL">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_l,         "IMAGE_SAMPLE_L">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_b,         "IMAGE_SAMPLE_B">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_b_cl,      "IMAGE_SAMPLE_B_CL">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_lz,        "IMAGE_SAMPLE_LZ">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_cd,        "IMAGE_SAMPLE_CD">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_cd_cl,     "IMAGE_SAMPLE_CD_CL">;
+
+// Sample with comparison
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c,         "IMAGE_SAMPLE_C">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_cl,      "IMAGE_SAMPLE_C_CL">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_d,       "IMAGE_SAMPLE_C_D">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_d_cl,    "IMAGE_SAMPLE_C_D_CL">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_l,       "IMAGE_SAMPLE_C_L">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_b,       "IMAGE_SAMPLE_C_B">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_b_cl,    "IMAGE_SAMPLE_C_B_CL">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_lz,      "IMAGE_SAMPLE_C_LZ">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_cd,      "IMAGE_SAMPLE_C_CD">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_cd_cl,   "IMAGE_SAMPLE_C_CD_CL">;
+
+// Sample with offsets
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_o,         "IMAGE_SAMPLE_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_cl_o,      "IMAGE_SAMPLE_CL_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_d_o,       "IMAGE_SAMPLE_D_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_d_cl_o,    "IMAGE_SAMPLE_D_CL_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_l_o,       "IMAGE_SAMPLE_L_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_b_o,       "IMAGE_SAMPLE_B_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_b_cl_o,    "IMAGE_SAMPLE_B_CL_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_lz_o,      "IMAGE_SAMPLE_LZ_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_cd_o,      "IMAGE_SAMPLE_CD_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_cd_cl_o,   "IMAGE_SAMPLE_CD_CL_O">;
+
+// Sample with comparison and offsets
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_o,       "IMAGE_SAMPLE_C_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_cl_o,    "IMAGE_SAMPLE_C_CL_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_d_o,     "IMAGE_SAMPLE_C_D_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_d_cl_o,  "IMAGE_SAMPLE_C_D_CL_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_l_o,     "IMAGE_SAMPLE_C_L_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_b_o,     "IMAGE_SAMPLE_C_B_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_b_cl_o,  "IMAGE_SAMPLE_C_B_CL_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_lz_o,    "IMAGE_SAMPLE_C_LZ_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_cd_o,    "IMAGE_SAMPLE_C_CD_O">;
+defm : AMDGCNSamplePatterns<int_amdgcn_image_sample_c_cd_cl_o, "IMAGE_SAMPLE_C_CD_CL_O">;
+
+// Gather opcodes
+// Only the variants which make sense are defined.
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4,           IMAGE_GATHER4_V4_V2,        v2f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4,           IMAGE_GATHER4_V4_V4,        v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_cl,        IMAGE_GATHER4_CL_V4_V4,     v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_l,         IMAGE_GATHER4_L_V4_V4,      v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_b,         IMAGE_GATHER4_B_V4_V4,      v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_b_cl,      IMAGE_GATHER4_B_CL_V4_V4,   v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_b_cl,      IMAGE_GATHER4_B_CL_V4_V8,   v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_lz,        IMAGE_GATHER4_LZ_V4_V2,     v2f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_lz,        IMAGE_GATHER4_LZ_V4_V4,     v4f32>;
+
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c,         IMAGE_GATHER4_C_V4_V4,      v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_cl,      IMAGE_GATHER4_C_CL_V4_V4,   v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_cl,      IMAGE_GATHER4_C_CL_V4_V8,   v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_l,       IMAGE_GATHER4_C_L_V4_V4,    v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_l,       IMAGE_GATHER4_C_L_V4_V8,    v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_b,       IMAGE_GATHER4_C_B_V4_V4,    v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_b,       IMAGE_GATHER4_C_B_V4_V8,    v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_b_cl,    IMAGE_GATHER4_C_B_CL_V4_V8, v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_lz,      IMAGE_GATHER4_C_LZ_V4_V4,   v4f32>;
+
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_o,         IMAGE_GATHER4_O_V4_V4,      v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_cl_o,      IMAGE_GATHER4_CL_O_V4_V4,   v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_cl_o,      IMAGE_GATHER4_CL_O_V4_V8,   v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_l_o,       IMAGE_GATHER4_L_O_V4_V4,    v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_l_o,       IMAGE_GATHER4_L_O_V4_V8,    v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_b_o,       IMAGE_GATHER4_B_O_V4_V4,    v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_b_o,       IMAGE_GATHER4_B_O_V4_V8,    v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_b_cl_o,    IMAGE_GATHER4_B_CL_O_V4_V8, v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_lz_o,      IMAGE_GATHER4_LZ_O_V4_V4,   v4f32>;
+
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_o,       IMAGE_GATHER4_C_O_V4_V4,    v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_o,       IMAGE_GATHER4_C_O_V4_V8,    v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_cl_o,    IMAGE_GATHER4_C_CL_O_V4_V8, v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_l_o,     IMAGE_GATHER4_C_L_O_V4_V8,  v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_b_o,     IMAGE_GATHER4_C_B_O_V4_V8,  v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_b_cl_o,  IMAGE_GATHER4_C_B_CL_O_V4_V8, v8f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_lz_o,    IMAGE_GATHER4_C_LZ_O_V4_V4, v4f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_gather4_c_lz_o,    IMAGE_GATHER4_C_LZ_O_V4_V8, v8f32>;
+
+def : AMDGCNSamplePattern<int_amdgcn_image_getlod, IMAGE_GET_LOD_V4_V1, f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_getlod, IMAGE_GET_LOD_V4_V2, v2f32>;
+def : AMDGCNSamplePattern<int_amdgcn_image_getlod, IMAGE_GET_LOD_V4_V4, v4f32>;
+
+// Image atomics
 defm : ImageAtomicPatterns<int_amdgcn_image_atomic_swap, "IMAGE_ATOMIC_SWAP">;
 def : ImageAtomicCmpSwapPattern<IMAGE_ATOMIC_CMPSWAP_V1, i32>;
 def : ImageAtomicCmpSwapPattern<IMAGE_ATOMIC_CMPSWAP_V2, v2i32>;
 def : ImageAtomicCmpSwapPattern<IMAGE_ATOMIC_CMPSWAP_V4, v4i32>;
 defm : ImageAtomicPatterns<int_amdgcn_image_atomic_add, "IMAGE_ATOMIC_ADD">;
 defm : ImageAtomicPatterns<int_amdgcn_image_atomic_sub, "IMAGE_ATOMIC_SUB">;
 defm : ImageAtomicPatterns<int_amdgcn_image_atomic_smin, "IMAGE_ATOMIC_SMIN">;
 defm : ImageAtomicPatterns<int_amdgcn_image_atomic_umin, "IMAGE_ATOMIC_UMIN">;
 defm : ImageAtomicPatterns<int_amdgcn_image_atomic_smax, "IMAGE_ATOMIC_SMAX">;
 defm : ImageAtomicPatterns<int_amdgcn_image_atomic_umax, "IMAGE_ATOMIC_UMAX">;

test/CodeGen/AMDGPU/llvm.amdgcn.image.gather4.ll

+; RUN: llc < %s -march=amdgcn -mcpu=verde -verify-machineinstrs | FileCheck --check-prefix=GCN %s
+; RUN: llc < %s -march=amdgcn -mcpu=tonga -verify-machineinstrs | FileCheck --check-prefix=GCN %s
+
+; GCN-LABEL: {{^}}gather4_v2:
+; GCN: image_gather4 {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_v2(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.v2f32(<2 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4:
+; GCN: image_gather4 {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_cl:
+; GCN: image_gather4_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.cl.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_l:
+; GCN: image_gather4_l {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_l(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.l.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_b:
+; GCN: image_gather4_b {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_b(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.b.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_b_cl:
+; GCN: image_gather4_b_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_b_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.b.cl.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_b_cl_v8:
+; GCN: image_gather4_b_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_b_cl_v8(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.b.cl.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_lz_v2:
+; GCN: image_gather4_lz {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_lz_v2(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.lz.v2f32(<2 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_lz:
+; GCN: image_gather4_lz {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_lz(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.lz.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+
+
+; GCN-LABEL: {{^}}gather4_o:
+; GCN: image_gather4_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_cl_o:
+; GCN: image_gather4_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_cl_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.cl.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_cl_o_v8:
+; GCN: image_gather4_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_cl_o_v8(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.cl.o.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_l_o:
+; GCN: image_gather4_l_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_l_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.l.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_l_o_v8:
+; GCN: image_gather4_l_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_l_o_v8(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.l.o.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_b_o:
+; GCN: image_gather4_b_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_b_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.b.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_b_o_v8:
+; GCN: image_gather4_b_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_b_o_v8(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.b.o.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_b_cl_o:
+; GCN: image_gather4_b_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_b_cl_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.b.cl.o.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_lz_o:
+; GCN: image_gather4_lz_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_lz_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.lz.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+
+; GCN-LABEL: {{^}}gather4_c:
+; GCN: image_gather4_c {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_cl:
+; GCN: image_gather4_c_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.cl.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_cl_v8:
+; GCN: image_gather4_c_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_cl_v8(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.cl.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_l:
+; GCN: image_gather4_c_l {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_l(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.l.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_l_v8:
+; GCN: image_gather4_c_l {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_l_v8(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.l.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_b:
+; GCN: image_gather4_c_b {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_b(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.b.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_b_v8:
+; GCN: image_gather4_c_b {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_b_v8(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.b.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_b_cl:
+; GCN: image_gather4_c_b_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_b_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.b.cl.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_lz:
+; GCN: image_gather4_c_lz {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_lz(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.lz.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+
+; GCN-LABEL: {{^}}gather4_c_o:
+; GCN: image_gather4_c_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_o_v8:
+; GCN: image_gather4_c_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_o_v8(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.o.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_cl_o:
+; GCN: image_gather4_c_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_cl_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.cl.o.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_l_o:
+; GCN: image_gather4_c_l_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_l_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.l.o.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_b_o:
+; GCN: image_gather4_c_b_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_b_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.b.o.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_b_cl_o:
+; GCN: image_gather4_c_b_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_b_cl_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.b.cl.o.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_lz_o:
+; GCN: image_gather4_c_lz_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_lz_o(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.lz.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}gather4_c_lz_o_v8:
+; GCN: image_gather4_c_lz_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0x1 da
+define void @gather4_c_lz_o_v8(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.gather4.c.lz.o.v8f32(<8 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 1, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+
+declare <4 x float> @llvm.amdgcn.image.gather4.v2f32(<2 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.cl.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.l.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.b.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.b.cl.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.b.cl.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.lz.v2f32(<2 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.lz.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+
+declare <4 x float> @llvm.amdgcn.image.gather4.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.cl.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.cl.o.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.l.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.l.o.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.b.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.b.o.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.b.cl.o.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.lz.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+
+declare <4 x float> @llvm.amdgcn.image.gather4.c.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.cl.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.cl.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.l.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.l.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.b.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.b.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.b.cl.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.lz.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+
+declare <4 x float> @llvm.amdgcn.image.gather4.c.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.o.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.cl.o.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.l.o.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.b.o.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.b.cl.o.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.lz.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.gather4.c.lz.o.v8f32(<8 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+
+
+attributes #0 = { nounwind readnone }

test/CodeGen/AMDGPU/llvm.amdgcn.image.getlod.ll

+; RUN: llc < %s -march=amdgcn -mcpu=verde -verify-machineinstrs | FileCheck --check-prefix=GCN %s
+; RUN: llc < %s -march=amdgcn -mcpu=tonga -verify-machineinstrs | FileCheck --check-prefix=GCN %s
+
+; GCN-LABEL: {{^}}getlod:
+; GCN: image_get_lod {{v\[[0-9]+:[0-9]+\]}}, {{v[0-9]+}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf da
+define void @getlod(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.getlod.f32(float undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}getlod_v2:
+; GCN: image_get_lod {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf da
+define void @getlod_v2(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.getlod.v2f32(<2 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out 
+  ret void
+}
+
+; GCN-LABEL: {{^}}getlod_v4:
+; GCN: image_get_lod {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf da
+define void @getlod_v4(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.getlod.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 1)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+
+declare <4 x float> @llvm.amdgcn.image.getlod.f32(float, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.getlod.v2f32(<2 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.getlod.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+
+
+attributes #0 = { nounwind readnone }

test/CodeGen/AMDGPU/llvm.amdgcn.image.ll

 ;CHECK: s_waitcnt vmcnt(0)
 ;CHECK: image_store v[0:3], v4, s[16:23] dmask:0xf unorm
 define amdgpu_ps void @image_store_wait(<8 x i32> inreg, <8 x i32> inreg, <8 x i32> inreg, <4 x float>, i32) {
 main_body:
   call void @llvm.amdgcn.image.store.i32(<4 x float> %3, i32 %4, <8 x i32> %0, i32 15, i1 0, i1 0, i1 0, i1 0)
   %data = call <4 x float> @llvm.amdgcn.image.load.i32(i32 %4, <8 x i32> %1, i32 15, i1 0, i1 0, i1 0, i1 0)
   call void @llvm.amdgcn.image.store.i32(<4 x float> %data, i32 %4, <8 x i32> %2, i32 15, i1 0, i1 0, i1 0, i1 0)
   ret void
 }
 
+;CHECK-LABEL: {{^}}getresinfo:
+;CHECK: image_get_resinfo {{v\[[0-9]+:[0-9]+\]}}, {{v[0-9]+}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define amdgpu_ps void @getresinfo() {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.getresinfo.i32(i32 undef, <8 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0)
+  %r0 = extractelement <4 x float> %r, i32 0
+  %r1 = extractelement <4 x float> %r, i32 1
+  %r2 = extractelement <4 x float> %r, i32 2
+  %r3 = extractelement <4 x float> %r, i32 3
+  call void @llvm.SI.export(i32 15, i32 1, i32 1, i32 0, i32 1, float %r0, float %r1, float %r2, float %r3)
+  ret void
+}
+
+
 declare void @llvm.amdgcn.image.store.i32(<4 x float>, i32, <8 x i32>, i32, i1, i1, i1, i1) #0
 declare void @llvm.amdgcn.image.store.v2i32(<4 x float>, <2 x i32>, <8 x i32>, i32, i1, i1, i1, i1) #0
 declare void @llvm.amdgcn.image.store.v4i32(<4 x float>, <4 x i32>, <8 x i32>, i32, i1, i1, i1, i1) #0
 declare void @llvm.amdgcn.image.store.mip.v4i32(<4 x float>, <4 x i32>, <8 x i32>, i32, i1, i1, i1, i1) #0
 
 declare <4 x float> @llvm.amdgcn.image.load.i32(i32, <8 x i32>, i32, i1, i1, i1, i1) #1
 declare <4 x float> @llvm.amdgcn.image.load.v2i32(<2 x i32>, <8 x i32>, i32, i1, i1, i1, i1) #1
 declare <4 x float> @llvm.amdgcn.image.load.v4i32(<4 x i32>, <8 x i32>, i32, i1, i1, i1, i1) #1
 declare <4 x float> @llvm.amdgcn.image.load.mip.v4i32(<4 x i32>, <8 x i32>, i32, i1, i1, i1, i1) #1
 
+declare <4 x float> @llvm.amdgcn.image.getresinfo.i32(i32, <8 x i32>, i32, i1, i1, i1, i1) #1
+declare void @llvm.SI.export(i32, i32, i32, i32, i32, float, float, float, float)
+
 attributes #0 = { nounwind }
 attributes #1 = { nounwind readonly }

test/CodeGen/AMDGPU/llvm.amdgcn.image.sample.ll

+; RUN: llc < %s -march=amdgcn -mcpu=verde -verify-machineinstrs | FileCheck --check-prefix=GCN %s
+; RUN: llc < %s -march=amdgcn -mcpu=tonga -verify-machineinstrs | FileCheck --check-prefix=GCN %s
+
+; GCN-LABEL: {{^}}sample:
+; GCN: image_sample {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_cl:
+; GCN: image_sample_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.cl.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_d:
+; GCN: image_sample_d {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_d(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.d.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_d_cl:
+; GCN: image_sample_d_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_d_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.d.cl.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_l:
+; GCN: image_sample_l {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_l(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.l.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_b:
+; GCN: image_sample_b {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_b(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.b.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out 
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_b_cl:
+; GCN: image_sample_b_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_b_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.b.cl.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_lz:
+; GCN: image_sample_lz {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_lz(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.lz.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_cd:
+; GCN: image_sample_cd {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_cd(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.cd.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_cd_cl:
+; GCN: image_sample_cd_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_cd_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.cd.cl.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c:
+; GCN: image_sample_c {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_cl:
+; GCN: image_sample_c_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.cl.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_d:
+; GCN: image_sample_c_d {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_d(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.d.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_d_cl:
+; GCN: image_sample_c_d_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_d_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.d.cl.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_l:
+; GCN: image_sample_c_l {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_l(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.l.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_b:
+; GCN: image_sample_c_b {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_b(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.b.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_b_cl:
+; GCN: image_sample_c_b_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_b_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.b.cl.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_lz:
+; GCN: image_sample_c_lz {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_lz(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.lz.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_cd:
+; GCN: image_sample_c_cd {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_cd(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.cd.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_cd_cl:
+; GCN: image_sample_c_cd_cl {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_cd_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.cd.cl.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+
+declare <4 x float> @llvm.amdgcn.image.sample.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.cl.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.d.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.d.cl.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.l.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.b.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.b.cl.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.lz.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.cd.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.cd.cl.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+
+declare <4 x float> @llvm.amdgcn.image.sample.c.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.cl.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.d.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.d.cl.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.l.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.b.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.b.cl.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.lz.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.cd.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.cd.cl.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+
+
+attributes #0 = { nounwind readnone }

test/CodeGen/AMDGPU/llvm.amdgcn.image.sample.o.ll

+; RUN: llc < %s -march=amdgcn -mcpu=verde -verify-machineinstrs | FileCheck --check-prefix=GCN %s
+; RUN: llc < %s -march=amdgcn -mcpu=tonga -verify-machineinstrs | FileCheck --check-prefix=GCN %s
+
+; GCN-LABEL: {{^}}sample:
+; GCN: image_sample_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_cl:
+; GCN: image_sample_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.cl.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_d:
+; GCN: image_sample_d_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_d(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.d.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_d_cl:
+; GCN: image_sample_d_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_d_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.d.cl.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_l:
+; GCN: image_sample_l_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_l(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.l.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_b:
+; GCN: image_sample_b_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_b(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.b.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_b_cl:
+; GCN: image_sample_b_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_b_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.b.cl.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_lz:
+; GCN: image_sample_lz_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_lz(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.lz.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_cd:
+; GCN: image_sample_cd_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_cd(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.cd.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_cd_cl:
+; GCN: image_sample_cd_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_cd_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.cd.cl.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c:
+; GCN: image_sample_c_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_cl:
+; GCN: image_sample_c_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.cl.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_d:
+; GCN: image_sample_c_d_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_d(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.d.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_d_cl:
+; GCN: image_sample_c_d_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_d_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.d.cl.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_l:
+; GCN: image_sample_c_l_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_l(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.l.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_b:
+; GCN: image_sample_c_b_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_b(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.b.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_b_cl:
+; GCN: image_sample_c_b_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_b_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.b.cl.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_lz:
+; GCN: image_sample_c_lz_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_lz(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.lz.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_cd:
+; GCN: image_sample_c_cd_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_cd(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.cd.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+; GCN-LABEL: {{^}}sample_c_cd_cl:
+; GCN: image_sample_c_cd_cl_o {{v\[[0-9]+:[0-9]+\]}}, {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}} dmask:0xf
+define void @sample_c_cd_cl(<4 x float> addrspace(1)* %out) {
+main_body:
+  %r = call <4 x float> @llvm.amdgcn.image.sample.c.cd.cl.o.v4f32(<4 x float> undef, <8 x i32> undef, <4 x i32> undef, i32 15, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0, i1 0)
+  store <4 x float> %r, <4 x float> addrspace(1)* %out
+  ret void
+}
+
+
+declare <4 x float> @llvm.amdgcn.image.sample.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.cl.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.d.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.d.cl.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.l.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.b.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.b.cl.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.lz.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.cd.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.cd.cl.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+
+declare <4 x float> @llvm.amdgcn.image.sample.c.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.cl.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.d.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.d.cl.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.l.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.b.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.b.cl.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.lz.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.cd.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+declare <4 x float> @llvm.amdgcn.image.sample.c.cd.cl.o.v4f32(<4 x float>, <8 x i32>, <4 x i32>, i32, i1, i1, i1, i1, i1, i1, i1) #0
+
+
+attributes #0 = { nounwind readnone }