GlobalISel: Support narrowing zextload/sextload

[aemerson]

lib/CodeGen/GlobalISel/LegalizerHelper.cpp

526

Is this really needed?

[arsenm]

lib/CodeGen/GlobalISel/LegalizerHelper.cpp

526

I’m not sure what you mean? Otherwise you’ll end up with an extload with the same source and dest size (which I think the verifier should reject)

[dsanders]

lib/CodeGen/GlobalISel/LegalizerHelper.cpp

526

I agree the verifier should reject it. 's32 = G_ZEXTLOAD ... :: (load 4 from ...)' isn't exactly wrong but it serves no purpose compared to 's32 = G_LOAD ... :: (load 4 from ...)'

540

However, this should be invalid for the same reason, 's32 = G_SEXT s32' doesn't make sense. It would be better for the case where it lowers to G_LOAD to just replace the opcode in place. If we have observers plumbed into the legalizer yet we should also call the changingInstr()+changedInstr() events.

lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp

210

This is redundant given that clampScalar() limits the type to exactly s32. It's nice for consistency but it costs a small amount of compile-time to perform the test.

[arsenm]

lib/CodeGen/GlobalISel/LegalizerHelper.cpp

540

I don't see why this would actually happen? I suppose that depends on what the constraints are for the specified legalize actions. I don't think you should be allow to specify narrow on a type to the same type. Assuming narrowing a load to the same type isn't allowed, this shouldn't be an issue.

[dsanders]

LGTM

lib/CodeGen/GlobalISel/LegalizerHelper.cpp

540

Sorry, I was thinking that if the code above selects the G_LOAD then we can end up with a G_[SZ]EXT that doesn't extend but that can only happen if the G_[SZ]EXTLOAD was invalid in the first place.

[arsenm]

r351856

lib/CodeGen/GlobalISel/LegalizerHelper.cpp

case TargetOpcode::G_LOAD: {
     unsigned DstReg = MI.getOperand(0).getReg();
     if(MRI.getType(DstReg).isVector())
       MIRBuilder.buildBuildVector(DstReg, DstRegs);
     else
       MIRBuilder.buildMerge(DstReg, DstRegs);
     MI.eraseFromParent();
     return Legalized;
   }
+  case TargetOpcode::G_ZEXTLOAD:
+  case TargetOpcode::G_SEXTLOAD: {
+    bool ZExt = MI.getOpcode() == TargetOpcode::G_ZEXTLOAD;
+    unsigned DstReg = MI.getOperand(0).getReg();
+    unsigned PtrReg = MI.getOperand(1).getReg();
+
+    unsigned TmpReg = MRI.createGenericVirtualRegister(NarrowTy);
+    auto &MMO = **MI.memoperands_begin();
+    if (MMO.getSize() * 8 == NarrowSize) {
+      MIRBuilder.buildLoad(TmpReg, PtrReg, MMO);
+    } else {
+      unsigned ExtLoad = ZExt ? TargetOpcode::G_ZEXTLOAD
+        : TargetOpcode::G_SEXTLOAD;
+      MIRBuilder.buildInstr(ExtLoad)
+        .addDef(TmpReg)
+        .addUse(PtrReg)
+        .addMemOperand(&MMO);
+    }
+
+    if (ZExt)
+      MIRBuilder.buildZExt(DstReg, TmpReg);
+    else
+      MIRBuilder.buildSExt(DstReg, TmpReg);
+
+    MI.eraseFromParent();
+    return Legalized;
+  }
   case TargetOpcode::G_STORE: {
     // FIXME: add support for when SizeOp0 isn't an exact multiple of
     // NarrowSize.
     if (SizeOp0 % NarrowSize != 0)
       return UnableToLegalize;
 
     const auto &MMO = **MI.memoperands_begin();
     // This implementation doesn't work for atomics. Give up instead of doing

lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp

.legalIf([=, &ST](const LegalityQuery &Query) {
         case 512:
           // TODO: constant loads
         default:
           return false;
         }
       });
 
 
+  auto &ExtLoads = getActionDefinitionsBuilder({G_SEXTLOAD, G_ZEXTLOAD})
+    .legalForTypesWithMemSize({
+        {S32, GlobalPtr, 8},
+        {S32, GlobalPtr, 16},
+        {S32, LocalPtr, 8},
+        {S32, LocalPtr, 16},
+        {S32, PrivatePtr, 8},
+        {S32, PrivatePtr, 16}});
+  if (ST.hasFlatAddressSpace()) {
+    ExtLoads.legalForTypesWithMemSize({{S32, FlatPtr, 8},
+                                       {S32, FlatPtr, 16}});
+  }
+
+  ExtLoads.clampScalar(0, S32, S32)
+          .widenScalarToNextPow2(0)
+          .unsupportedIfMemSizeNotPow2()
+          .lower();
+
   auto &Atomics = getActionDefinitionsBuilder(
     {G_ATOMICRMW_XCHG, G_ATOMICRMW_ADD, G_ATOMICRMW_SUB,
      G_ATOMICRMW_AND, G_ATOMICRMW_OR, G_ATOMICRMW_XOR,
      G_ATOMICRMW_MAX, G_ATOMICRMW_MIN, G_ATOMICRMW_UMAX,
      G_ATOMICRMW_UMIN, G_ATOMIC_CMPXCHG})
     .legalFor({{S32, GlobalPtr}, {S32, LocalPtr},
                {S64, GlobalPtr}, {S64, LocalPtr}});
   if (ST.hasFlatAddressSpace()) {

test/CodeGen/AMDGPU/GlobalISel/legalize-sextload-flat.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=fiji -run-pass=legalizer -o - %s | FileCheck %s
+# RUN: not llc -mtriple=amdgcn-mesa-mesa3d -mcpu=tahiti -run-pass=legalizer -o /dev/null %s 2>&1 | FileCheck -check-prefix=ERROR %s
+
+# ERROR: LLVM ERROR: unable to legalize instruction: %2:_(s8) = G_LOAD %0:_(p0) :: (load 1) (in function: test_sextload_flat_i32_i8)
+
+---
+name: test_sextload_flat_i32_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_flat_i32_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p0) :: (load 1)
+    ; CHECK: $vgpr0 = COPY [[SEXTLOAD]](s32)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s32) = G_SEXTLOAD %0 :: (load 1, addrspace 0)
+    $vgpr0 = COPY %1
+...
+---
+name: test_sextload_flat_i32_i16
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_flat_i32_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p0) :: (load 2)
+    ; CHECK: $vgpr0 = COPY [[SEXTLOAD]](s32)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s32) = G_SEXTLOAD %0 :: (load 2, addrspace 0)
+    $vgpr0 = COPY %1
+...
+---
+name: test_sextload_flat_i31_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_flat_i31_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p0) :: (load 1)
+    ; CHECK: [[COPY1:%[0-9]+]]:_(s32) = COPY [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0 = COPY [[COPY1]](s32)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s31) = G_SEXTLOAD %0 :: (load 1, addrspace 0)
+    %2:_(s32) = G_ANYEXT %1
+    $vgpr0 = COPY %2
+...
+---
+name: test_sextload_flat_i64_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_flat_i64_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p0) :: (load 1)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 1, addrspace 0)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_sextload_flat_i64_i16
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_flat_i64_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p0) :: (load 2)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 2, addrspace 0)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_sextload_flat_i64_i32
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_flat_i64_i32
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[LOAD:%[0-9]+]]:_(s32) = G_LOAD [[COPY]](p0) :: (load 4)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[LOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 4, addrspace 0)
+    $vgpr0_vgpr1 = COPY %1
+...
+
+

test/CodeGen/AMDGPU/GlobalISel/legalize-sextload-global.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=fiji -run-pass=legalizer -o - %s | FileCheck %s
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=tahiti -run-pass=legalizer -o - %s | FileCheck %s
+
+---
+name: test_sextload_global_i32_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_global_i32_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p1) :: (load 1, addrspace 1)
+    ; CHECK: $vgpr0 = COPY [[SEXTLOAD]](s32)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s32) = G_SEXTLOAD %0 :: (load 1, addrspace 1)
+    $vgpr0 = COPY %1
+...
+---
+name: test_sextload_global_i32_i16
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_global_i32_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p1) :: (load 2, addrspace 1)
+    ; CHECK: $vgpr0 = COPY [[SEXTLOAD]](s32)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s32) = G_SEXTLOAD %0 :: (load 2, addrspace 1)
+    $vgpr0 = COPY %1
+...
+---
+name: test_sextload_global_i31_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_global_i31_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p1) :: (load 1, addrspace 1)
+    ; CHECK: [[COPY1:%[0-9]+]]:_(s32) = COPY [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0 = COPY [[COPY1]](s32)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s31) = G_SEXTLOAD %0 :: (load 1, addrspace 1)
+    %2:_(s32) = G_ANYEXT %1
+    $vgpr0 = COPY %2
+...
+---
+name: test_sextload_global_i64_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_global_i64_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p1) :: (load 1, addrspace 1)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 1, addrspace 1)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_sextload_global_i64_i16
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_global_i64_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p1) :: (load 2, addrspace 1)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 2, addrspace 1)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_sextload_global_i64_i32
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_global_i64_i32
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[LOAD:%[0-9]+]]:_(s32) = G_LOAD [[COPY]](p1) :: (load 4, addrspace 1)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[LOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 4, addrspace 1)
+    $vgpr0_vgpr1 = COPY %1
+...
+

test/CodeGen/AMDGPU/GlobalISel/legalize-sextload-local.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=fiji -run-pass=legalizer -o - %s | FileCheck %s
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=tahiti -run-pass=legalizer -o - %s | FileCheck %s
+---
+name: test_sextload_local_i32_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_sextload_local_i32_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p3) :: (load 1, addrspace 3)
+    ; CHECK: $vgpr0 = COPY [[SEXTLOAD]](s32)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s32) = G_SEXTLOAD %0 :: (load 1, addrspace 3)
+    $vgpr0 = COPY %1
+...
+---
+name: test_sextload_local_i32_i16
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_sextload_local_i32_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p3) :: (load 2, addrspace 3)
+    ; CHECK: $vgpr0 = COPY [[SEXTLOAD]](s32)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s32) = G_SEXTLOAD %0 :: (load 2, addrspace 3)
+    $vgpr0 = COPY %1
+...
+---
+name: test_sextload_local_i31_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_sextload_local_i31_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p3) :: (load 1, addrspace 3)
+    ; CHECK: [[COPY1:%[0-9]+]]:_(s32) = COPY [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0 = COPY [[COPY1]](s32)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s31) = G_SEXTLOAD %0 :: (load 1, addrspace 3)
+    %2:_(s32) = G_ANYEXT %1
+    $vgpr0 = COPY %2
+...
+---
+name: test_sextload_local_i64_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_sextload_local_i64_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p3) :: (load 1, addrspace 3)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 1, addrspace 3)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_sextload_local_i64_i16
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_sextload_local_i64_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p3) :: (load 2, addrspace 3)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 2, addrspace 3)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_sextload_local_i64_i32
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_local_i64_i32
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[LOAD:%[0-9]+]]:_(s32) = G_LOAD [[COPY]](p3) :: (load 4, addrspace 3)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[LOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 4, addrspace 3)
+    $vgpr0_vgpr1 = COPY %1
+...
+

test/CodeGen/AMDGPU/GlobalISel/legalize-sextload-private.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=fiji -run-pass=legalizer -o - %s | FileCheck %s
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=tahiti -run-pass=legalizer -o - %s | FileCheck %s
+
+---
+name: test_sextload_private_i32_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_sextload_private_i32_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p5) :: (load 1, addrspace 5)
+    ; CHECK: $vgpr0 = COPY [[SEXTLOAD]](s32)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s32) = G_SEXTLOAD %0 :: (load 1, addrspace 5)
+
+    $vgpr0 = COPY %1
+...
+---
+name: test_sextload_private_i32_i16
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_sextload_private_i32_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p5) :: (load 2, addrspace 5)
+    ; CHECK: $vgpr0 = COPY [[SEXTLOAD]](s32)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s32) = G_SEXTLOAD %0 :: (load 2, addrspace 5)
+    $vgpr0 = COPY %1
+...
+---
+name: test_sextload_private_i31_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_sextload_private_i31_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p5) :: (load 1, addrspace 5)
+    ; CHECK: [[COPY1:%[0-9]+]]:_(s32) = COPY [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0 = COPY [[COPY1]](s32)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s31) = G_SEXTLOAD %0 :: (load 1, addrspace 5)
+    %2:_(s32) = G_ANYEXT %1
+    $vgpr0 = COPY %2
+...
+---
+name: test_sextload_private_i64_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_sextload_private_i64_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p5) :: (load 1, addrspace 5)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 1, addrspace 5)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_sextload_private_i64_i16
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_sextload_private_i64_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[COPY]](p5) :: (load 2, addrspace 5)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[SEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 2, addrspace 5)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_sextload_private_i64_i32
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_sextload_private_i64_i32
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[LOAD:%[0-9]+]]:_(s32) = G_LOAD [[COPY]](p5) :: (load 4, addrspace 5)
+    ; CHECK: [[SEXT:%[0-9]+]]:_(s64) = G_SEXT [[LOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[SEXT]](s64)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s64) = G_SEXTLOAD %0 :: (load 4, addrspace 5)
+    $vgpr0_vgpr1 = COPY %1
+...
+

test/CodeGen/AMDGPU/GlobalISel/legalize-zextload-flat.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=fiji -run-pass=legalizer -o - %s | FileCheck %s
+# RUN: not llc -mtriple=amdgcn-mesa-mesa3d -mcpu=tahiti -run-pass=legalizer -o /dev/null %s 2>&1 | FileCheck -check-prefix=ERROR %s
+
+# ERROR: LLVM ERROR: unable to legalize instruction: %2:_(s8) = G_LOAD %0:_(p0) :: (load 1) (in function: test_zextload_flat_i32_i8)
+
+---
+name: test_zextload_flat_i32_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_flat_i32_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p0) :: (load 1)
+    ; CHECK: $vgpr0 = COPY [[ZEXTLOAD]](s32)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s32) = G_ZEXTLOAD %0 :: (load 1, addrspace 0)
+    $vgpr0 = COPY %1
+...
+---
+name: test_zextload_flat_i32_i16
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_flat_i32_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p0) :: (load 2)
+    ; CHECK: $vgpr0 = COPY [[ZEXTLOAD]](s32)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s32) = G_ZEXTLOAD %0 :: (load 2, addrspace 0)
+    $vgpr0 = COPY %1
+...
+---
+name: test_zextload_flat_i31_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_flat_i31_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p0) :: (load 1)
+    ; CHECK: [[COPY1:%[0-9]+]]:_(s32) = COPY [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0 = COPY [[COPY1]](s32)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s31) = G_ZEXTLOAD %0 :: (load 1, addrspace 0)
+    %2:_(s32) = G_ANYEXT %1
+    $vgpr0 = COPY %2
+...
+---
+name: test_zextload_flat_i64_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_flat_i64_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p0) :: (load 1)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 1, addrspace 0)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_zextload_flat_i64_i16
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_flat_i64_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p0) :: (load 2)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 2, addrspace 0)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_zextload_flat_i64_i32
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_flat_i64_i32
+    ; CHECK: [[COPY:%[0-9]+]]:_(p0) = COPY $vgpr0_vgpr1
+    ; CHECK: [[LOAD:%[0-9]+]]:_(s32) = G_LOAD [[COPY]](p0) :: (load 4)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[LOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p0) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 4, addrspace 0)
+    $vgpr0_vgpr1 = COPY %1
+...
+
+

test/CodeGen/AMDGPU/GlobalISel/legalize-zextload-global.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=fiji -run-pass=legalizer -o - %s | FileCheck %s
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=tahiti -run-pass=legalizer -o - %s | FileCheck %s
+
+---
+name: test_zextload_global_i32_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_global_i32_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p1) :: (load 1, addrspace 1)
+    ; CHECK: $vgpr0 = COPY [[ZEXTLOAD]](s32)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s32) = G_ZEXTLOAD %0 :: (load 1, addrspace 1)
+    $vgpr0 = COPY %1
+...
+---
+name: test_zextload_global_i32_i16
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_global_i32_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p1) :: (load 2, addrspace 1)
+    ; CHECK: $vgpr0 = COPY [[ZEXTLOAD]](s32)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s32) = G_ZEXTLOAD %0 :: (load 2, addrspace 1)
+    $vgpr0 = COPY %1
+...
+---
+name: test_zextload_global_i31_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_global_i31_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p1) :: (load 1, addrspace 1)
+    ; CHECK: [[COPY1:%[0-9]+]]:_(s32) = COPY [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0 = COPY [[COPY1]](s32)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s31) = G_ZEXTLOAD %0 :: (load 1, addrspace 1)
+    %2:_(s32) = G_ANYEXT %1
+    $vgpr0 = COPY %2
+...
+---
+name: test_zextload_global_i64_i8
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_global_i64_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p1) :: (load 1, addrspace 1)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 1, addrspace 1)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_zextload_global_i64_i16
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_global_i64_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p1) :: (load 2, addrspace 1)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 2, addrspace 1)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_zextload_global_i64_i32
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_global_i64_i32
+    ; CHECK: [[COPY:%[0-9]+]]:_(p1) = COPY $vgpr0_vgpr1
+    ; CHECK: [[LOAD:%[0-9]+]]:_(s32) = G_LOAD [[COPY]](p1) :: (load 4, addrspace 1)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[LOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p1) = COPY $vgpr0_vgpr1
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 4, addrspace 1)
+    $vgpr0_vgpr1 = COPY %1
+...
+

test/CodeGen/AMDGPU/GlobalISel/legalize-zextload-local.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=fiji -run-pass=legalizer -o - %s | FileCheck %s
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=tahiti -run-pass=legalizer -o - %s | FileCheck %s
+---
+name: test_zextload_local_i32_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_zextload_local_i32_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p3) :: (load 1, addrspace 3)
+    ; CHECK: $vgpr0 = COPY [[ZEXTLOAD]](s32)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s32) = G_ZEXTLOAD %0 :: (load 1, addrspace 3)
+    $vgpr0 = COPY %1
+...
+---
+name: test_zextload_local_i32_i16
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_zextload_local_i32_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p3) :: (load 2, addrspace 3)
+    ; CHECK: $vgpr0 = COPY [[ZEXTLOAD]](s32)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s32) = G_ZEXTLOAD %0 :: (load 2, addrspace 3)
+    $vgpr0 = COPY %1
+...
+---
+name: test_zextload_local_i31_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_zextload_local_i31_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p3) :: (load 1, addrspace 3)
+    ; CHECK: [[COPY1:%[0-9]+]]:_(s32) = COPY [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0 = COPY [[COPY1]](s32)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s31) = G_ZEXTLOAD %0 :: (load 1, addrspace 3)
+    %2:_(s32) = G_ANYEXT %1
+    $vgpr0 = COPY %2
+...
+---
+name: test_zextload_local_i64_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_zextload_local_i64_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p3) :: (load 1, addrspace 3)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 1, addrspace 3)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_zextload_local_i64_i16
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_zextload_local_i64_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p3) :: (load 2, addrspace 3)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 2, addrspace 3)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_zextload_local_i64_i32
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_local_i64_i32
+    ; CHECK: [[COPY:%[0-9]+]]:_(p3) = COPY $vgpr0
+    ; CHECK: [[LOAD:%[0-9]+]]:_(s32) = G_LOAD [[COPY]](p3) :: (load 4, addrspace 3)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[LOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p3) = COPY $vgpr0
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 4, addrspace 3)
+    $vgpr0_vgpr1 = COPY %1
+...
+

test/CodeGen/AMDGPU/GlobalISel/legalize-zextload-private.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=fiji -run-pass=legalizer -o - %s | FileCheck %s
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=tahiti -run-pass=legalizer -o - %s | FileCheck %s
+
+---
+name: test_zextload_private_i32_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_zextload_private_i32_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p5) :: (load 1, addrspace 5)
+    ; CHECK: $vgpr0 = COPY [[ZEXTLOAD]](s32)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s32) = G_ZEXTLOAD %0 :: (load 1, addrspace 5)
+    $vgpr0 = COPY %1
+...
+---
+name: test_zextload_private_i32_i16
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_zextload_private_i32_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p5) :: (load 2, addrspace 5)
+    ; CHECK: $vgpr0 = COPY [[ZEXTLOAD]](s32)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s32) = G_ZEXTLOAD %0 :: (load 2, addrspace 5)
+    $vgpr0 = COPY %1
+...
+---
+name: test_zextload_private_i31_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_zextload_private_i31_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p5) :: (load 1, addrspace 5)
+    ; CHECK: [[COPY1:%[0-9]+]]:_(s32) = COPY [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0 = COPY [[COPY1]](s32)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s31) = G_ZEXTLOAD %0 :: (load 1, addrspace 5)
+    %2:_(s32) = G_ANYEXT %1
+    $vgpr0 = COPY %2
+...
+---
+name: test_zextload_private_i64_i8
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_zextload_private_i64_i8
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p5) :: (load 1, addrspace 5)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 1, addrspace 5)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_zextload_private_i64_i16
+body: |
+  bb.0:
+    liveins: $vgpr0
+
+    ; CHECK-LABEL: name: test_zextload_private_i64_i16
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[ZEXTLOAD:%[0-9]+]]:_(s32) = G_ZEXTLOAD [[COPY]](p5) :: (load 2, addrspace 5)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[ZEXTLOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 2, addrspace 5)
+    $vgpr0_vgpr1 = COPY %1
+...
+---
+name: test_zextload_private_i64_i32
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1
+
+    ; CHECK-LABEL: name: test_zextload_private_i64_i32
+    ; CHECK: [[COPY:%[0-9]+]]:_(p5) = COPY $vgpr0
+    ; CHECK: [[LOAD:%[0-9]+]]:_(s32) = G_LOAD [[COPY]](p5) :: (load 4, addrspace 5)
+    ; CHECK: [[ZEXT:%[0-9]+]]:_(s64) = G_ZEXT [[LOAD]](s32)
+    ; CHECK: $vgpr0_vgpr1 = COPY [[ZEXT]](s64)
+    %0:_(p5) = COPY $vgpr0
+    %1:_(s64) = G_ZEXTLOAD %0 :: (load 4, addrspace 5)
+    $vgpr0_vgpr1 = COPY %1
+...
+