Add a verifier test to check the access on both sides of COPY are the same

[aditya_nandakumar]

Summary

Only checks if there are no sub registers involved for now.
There are lots of failures currently which need to be fixed by various backends.

Splitting this off from https://reviews.llvm.org/D37640

[igorb]

Failed to build as a shared library -DBUILD_SHARED_LIBS=1

[igorb]

lib/CodeGen/MachineVerifier.cpp

982

For X86 back-end this is not always correct.
I think in case a few register classes with different size are mapped to the same physical registers, it is impossible to identify when the COPY is illegal without some additional target specific info.

def FR32 : RegisterClass<"X86", [f32], 32, (sequence "XMM%u", 0, 15)>;
def VR128 : RegisterClass<"X86", [v4f32, v2f64, v16i8, v8i16, v4i32, v2i64], 128, (add FR32)>;

For example the test fail DAG/GISEL (with the patch)
./bin/llc -O0 -mtriple=x86_64-linux-gnu -verify-machineinstrs -stop-before=legalizer
./bin/llc -O0 -mtriple=x86_64-linux-gnu -global-isel -verify-machineinstrs -stop-before=legalizer

define float @test_add_float(float %arg1, float %arg2) {
  %ret = fadd float %arg1, %arg2
  ret float %ret
}

for the correct MIR. (xmm size is 128)

%0(s32) = COPY %xmm0
%1(s32) = COPY %xmm1
%2(s32) = G_FADD %0, %1
%xmm0 = COPY %2(s32)

Thanks

[qcolombet]

lib/CodeGen/MachineVerifier.cpp

982

I'd argue that when generic types are on one side of a copy, i.e., this is a pre-isel copy, we want the size to exactly match.
Thus, if the target wants to do truncate and whatnot at this point, it needs to use G_TRUNC, G_ANYEXT, and so on.

That being said, the check is too broad. We need to check that only for pre-isel copy. Regular copies require indeed target knowledge (e.g., eflags = copy gpr is valid on x86).

[aditya_nandakumar]

lib/CodeGen/MachineVerifier.cpp

982

I can update the check to do it only when there are generic types involved (ie one of src and dst has a generic virtual register).

[qcolombet]

lib/CodeGen/MachineVerifier.cpp

982

That would be perfect.

[aditya_nandakumar]

Updated to check only if one of Src/Dst has generic virtual registers.

[aditya_nandakumar]

Updated to refactor getRegSizeInBits into TargetRegisterInfo. There are 57 failures in total.

[qcolombet]

LGTM with one suggestion

lib/CodeGen/MachineVerifier.cpp

986

Seeing this pattern being actually the implementation of RegisterBankInfo::getSizeInBits, I would suggest to move the whole RegisterBankInfo::getSizeInBits directly in TargetRegisterInfo.

[aditya_nandakumar]

Also, there are various tests that need to be updated by the various backends, and I can only land once those are updated. I haven't had time to update all the backends.

lib/CodeGen/MachineVerifier.cpp

986

I thought TargetRegisterInfo doesn't have LLT headers pulled in (but looks like it is getting transitively pulled in). I can pretty much move the implementation of RegisterBankInfo::getSizeInBits into TargetRegisterInfo.

[aditya_nandakumar]

moved getSizeInBits to TargetRegisterInfo as suggested by Quentin.

[aditya_nandakumar]

Updated AArch64 tests.

Would it be possible for someone to update the ARM/X86 tests?

[rovka]

Sorry, I just noticed this. I'll have a look at the ARM side of things.

[rovka]

Hi, here's the patch for the ARM GlobalISel tests: https://reviews.llvm.org/differential/diff/119563/
Let me know if there's anything else that needs addressing on ARM.

[aditya_nandakumar]

Thanks Diana.
@igorb would it be possible for you to look at X86 when you get a chance? If not, I can update it.

[zvi]

Hi All,

I will try to cover for Igor in helping with the X86 part of this patch.

AFAIU, the challenge is in getting the call/return lowering working for the case where a function's argument/return-value is a f32 or a f64. Some subtargets will use the FP stack registers (which are 80-bit wide), and other will use XMM's (which are 128-bit wide).
Example:

define float @foo(float %arg1, float %arg2) {
  ret float %arg2
}

Existing state on ToT: llc -mtriple=x86_64-linux-gnu -mattr=+sse2 -global-isel -stop-after=irtranslator

body:             |
  bb.1 (%ir-block.0):
    liveins: %xmm0, %xmm1

    %0:_(s32) = COPY %xmm0
    %1:_(s32) = COPY %xmm1
    %xmm0 = COPY %1(s32)
    RET 0, implicit %xmm0

Here are some options for resolving this issue:

1. Perform the full expansion in X86CallLowering using G_COPY/G_TRUNC and G_ANYEXT/G_COPY. Fix whatever needs fixing to make it work.
2. X86CallLoweing will continue with generation of G_COPY's with incompatible src/dest bit-sizes, but will assign register classes to the virtual registers. This will technically work because the verification check is guarded by the condition 'SrcTy.isValid() || DstTy.isValid()', which will not be met if final register classes are assigned.
3. Introduce an X86-specific COPY instruction which will be pattern-matched in Instruction-Selection. This instruction will be generated instead of G_COPY's.

I haven't tried implementing any of the above options, and given my limited knowledge in GlobalISel, would appreciate any feedback on this.

Thanks, Zvi

[aditya_nandakumar]

I would definitely prefer the sizes to be right coming out of the CallLowering - so option #1.

[aditya_nandakumar]

ping?

[igorb]

Hi Aditya ,
Unfortunately, I don't have time to work on this task

Regards,
Igor

[aivchenk]

Hi Aditya,
I'm going to take this over from Igor; Will do that in Jan

[rtereshin]

lib/CodeGen/MachineVerifier.cpp

985

What if neither of the registers has a class? That would mean, both registers have LLTs. Then, the check won't complain if those LLTs have the same size in bits, but don't exactly match, like v2s32 and v4s16, for instance.

Shouldn't LLTs match exactly if both are valid?

[aditya_nandakumar]

lib/CodeGen/MachineVerifier.cpp

985

Good catch. I should be checking LLTs are equal here. I will update the patch.

[aivchenk]

This comment has been deleted.

lib/CodeGen/MachineVerifier.cpp

977

Here we rely on having operands in MI, while previously in MachineVerifier::visitMachineInstrBefore we could discover that there are not enough operands given. Should we guard that with foundErrors?

test/CodeGen/MIR/X86/machine-verifier.mir
fails due to that reason

[qcolombet]

lib/CodeGen/MachineVerifier.cpp

977

Sounds sensible to me.

lib/CodeGen/TargetRegisterInfo.cpp

429

This function is not consistent with RegisterBankInfo::getSizeInBits in the priority it gives to the different size when several sources are available (RC, Type).

RBI goes Type if any then RC, this one does the opposite and I believe this is not what we want.

At the very least, RBI should use this implementation instead of duplicating the logic.

[aivchenk]

X86 currently had/has three kinds of fails after this patch:

1. test/CodeGen/MIR/X86/machine-verifier.mir
   • should be fixed in this patch
2. Proper G_COPY/G_TRUNC and G_ANYEXT/G_COPY for floating point arguments
   • D42287 fixes that. However, it relies on getRegSizeInBits from this patch. We can either decide to split this patch or to submit it first and D42287 as the next one. The second option is not really good, though
3. The rest of the tests just needed small tweaks or reautogeneration. This is fixed and submitted in rL323209

[aditya_nandakumar]

Updated based on Quentin's/Roman's feedback.
With this patch, only X86 backend fails.

[qcolombet]

LGTM, coordinate with the x86 changes and you should be good to go!

[aditya_nandakumar]

I can push the getSizeInbits change in a NFC change all by itself, and then the verifier change can land separately once X86 finishes it?

lib/CodeGen/TargetRegisterInfo.cpp

429

Good catch. Will fix this.

[aditya_nandakumar]

Pushed the NFC change in r324125.

[aivchenk]

X86 part is submitted. All ready to go :)

include/llvm/Target/TargetRegisterInfo.h

virtual unsigned getRegPressureSetScore(const MachineFunction &MF,
                                           unsigned PSetID) const {
     return PSetID;
   }
 
   /// Get the weight in units of pressure for this register class.
   virtual const RegClassWeight &getRegClassWeight(
     const TargetRegisterClass *RC) const = 0;
 
+  /// Returns size in bits of a phys/virtual/generic register.
+  unsigned getRegSizeInBits(unsigned Reg, const MachineRegisterInfo &MRI) const;
+
   /// Get the weight in units of pressure for this register unit.
   virtual unsigned getRegUnitWeight(unsigned RegUnit) const = 0;
 
   /// Get the number of dimensions of register pressure.
   virtual unsigned getNumRegPressureSets() const = 0;
 
   /// Get the name of this register unit pressure set.
   virtual const char *getRegPressureSetName(unsigned Idx) const = 0;

lib/CodeGen/GlobalISel/RegisterBankInfo.cpp

for (unsigned OpIdx = 0,
     }
     DEBUG(dbgs() << '\n');
   }
 }
 
 unsigned RegisterBankInfo::getSizeInBits(unsigned Reg,
                                          const MachineRegisterInfo &MRI,
                                          const TargetRegisterInfo &TRI) {
-  const TargetRegisterClass *RC = nullptr;
-  if (TargetRegisterInfo::isPhysicalRegister(Reg)) {
-    // The size is not directly available for physical registers.
-    // Instead, we need to access a register class that contains Reg and
-    // get the size of that register class.
-    RC = TRI.getMinimalPhysRegClass(Reg);
-  } else {
-    LLT Ty = MRI.getType(Reg);
-    unsigned RegSize = Ty.isValid() ? Ty.getSizeInBits() : 0;
-    // If Reg is not a generic register, query the register class to
-    // get its size.
-    if (RegSize)
-      return RegSize;
-    // Since Reg is not a generic register, it must have a register class.
-    RC = MRI.getRegClass(Reg);
-  }
-  assert(RC && "Unable to deduce the register class");
-  return TRI.getRegSizeInBits(*RC);
+  return TRI.getRegSizeInBits(Reg, MRI);
 }
 
 //------------------------------------------------------------------------------
 // Helper classes implementation.
 //------------------------------------------------------------------------------
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
 LLVM_DUMP_METHOD void RegisterBankInfo::PartialMapping::dump() const {
   print(dbgs());

lib/CodeGen/MachineVerifier.cpp

if (!DstTy.isValid() ||
                          return false;
                        return true;
                      }))
       report("Generic Instruction G_PHI has operands with incompatible/missing "
              "types",
              MI);
     break;
   }
+  case TargetOpcode::COPY: {
+    const MachineOperand &DstOp = MI->getOperand(0);
+    const MachineOperand &SrcOp = MI->getOperand(1);
+    LLT DstTy = MRI->getType(DstOp.getReg());
+    LLT SrcTy = MRI->getType(SrcOp.getReg());
+    if (SrcTy.isValid() || DstTy.isValid()) {
+      unsigned SrcSize = TRI->getRegSizeInBits(SrcOp.getReg(), *MRI);
+      unsigned DstSize = TRI->getRegSizeInBits(DstOp.getReg(), *MRI);
+      assert(SrcSize && "Expecting size here");
+      assert(DstSize && "Expecting size here");
+      if (SrcSize != DstSize)
+        // Catch only obvious cases not involving subregs for now.
+        if (!DstOp.getSubReg() && !SrcOp.getSubReg()) {
+          report("Copy Instruction is illegal with mismatching sizes", MI);
+          errs() << "Def Size = " << DstSize << ", Src Size = " << SrcSize
+                 << "\n";
+        }
+    }
+    break;
+  }
   case TargetOpcode::STATEPOINT:
     if (!MI->getOperand(StatepointOpers::IDPos).isImm() ||
         !MI->getOperand(StatepointOpers::NBytesPos).isImm() ||
         !MI->getOperand(StatepointOpers::NCallArgsPos).isImm())
       report("meta operands to STATEPOINT not constant!", MI);
     break;
 
     auto VerifyStackMapConstant = [&](unsigned Offset) {

lib/CodeGen/TargetRegisterInfo.cpp

bool TargetRegisterInfo::regmaskSubsetEqual(const uint32_t *mask0,
                                             const uint32_t *mask1) const {
   unsigned N = (getNumRegs()+31) / 32;
   for (unsigned I = 0; I < N; ++I)
     if ((mask0[I] & mask1[I]) != mask0[I])
       return false;
   return true;
 }
 
+unsigned
+TargetRegisterInfo::getRegSizeInBits(unsigned Reg,
+                                     const MachineRegisterInfo &MRI) const {
+  const TargetRegisterClass *RC = nullptr;
+  if (isPhysicalRegister(Reg))
+    RC = getMinimalPhysRegClass(Reg);
+  else
+    RC = MRI.getRegClassOrNull(Reg);
+  if (!RC) {
+    LLT Ty = MRI.getType(Reg);
+    assert(Ty.isValid() && "Expecting valid type here");
+    return Ty.getSizeInBits();
+  }
+  return getRegSizeInBits(*RC);
+}
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
 LLVM_DUMP_METHOD
 void TargetRegisterInfo::dumpReg(unsigned Reg, unsigned SubRegIndex,
                                  const TargetRegisterInfo *TRI) {
   dbgs() << PrintReg(Reg, TRI, SubRegIndex) << "\n";
 }
 #endif

test/CodeGen/AArch64/GlobalISel/legalize-inserts.mir

 ---
 name:            test_inserts_4
 body: |
   bb.0:
     liveins: %w0
 
       ; A narrow insert gets surrounded by a G_ANYEXT/G_TRUNC pair.
     ; CHECK-LABEL: name: test_inserts_4
-    ; CHECK: [[VALEXT:%[0-9]+]](s32) = G_ANYEXT %1(s8)
-    ; CHECK: [[VAL:%[0-9]+]](s32) = G_INSERT [[VALEXT]], %0(s1), 0
-    ; CHECK: %3(s8) = G_TRUNC [[VAL]](s32)
-    %0:_(s1) = COPY %w0
-    %1:_(s8) = COPY %w1
+    ; CHECK: [[VALEXT:%[0-9]+]](s32) = COPY %2(s32)
+    ; CHECK: [[VAL:%[0-9]+]](s32) = G_INSERT [[VALEXT]], %1(s1), 0
+    ; CHECK: %5(s8) = G_TRUNC [[VAL]](s32)
+    %4:_(s32) = COPY %w0
+    %0:_(s1) = G_TRUNC %4
+    %5:_(s32) = COPY %w1
+    %1:_(s8) = G_TRUNC %5
     %2:_(p0) = COPY %x2
     %3:_(s8) = G_INSERT %1(s8), %0(s1), 0
     G_STORE %3(s8), %2(p0) :: (store 1)
     RET_ReallyLR
 ...
 
 ---
 name:            test_inserts_5

test/CodeGen/AArch64/GlobalISel/legalize-phi.mir

registers:
   - { id: 3, class: _, preferred-register: '' }
   - { id: 4, class: _, preferred-register: '' }
   - { id: 5, class: _, preferred-register: '' }
 liveins:
 body:             |
   bb.1:
    ; CHECK-LABEL: name: legalize_phi_ptr
    ; CHECK-LABEL: bb.0:
-   ; CHECK: %0(p0) = COPY %x0
-   ; CHECK: %1(p0) = COPY %x1
-   ; CHECK: %2(s1) = COPY %w2
+   ; CHECK: [[A:%[0-9]+]](p0) = COPY %x0
+   ; CHECK: [[B:%[0-9]+]](p0) = COPY %x1
+   ; CHECK: [[CE:%[0-9]+]](s32) = COPY %w2
+   ; CHECK: [[C:%[0-9]+]](s1) = G_TRUNC [[CE]]
 
    ; CHECK-LABEL: bb.1:
    ; CHECK-LABEL: bb.2:
-   ; CHECK: %3(p0) = G_PHI %0(p0), %bb.0, %1(p0), %bb.1
+   ; CHECK: %3(p0) = G_PHI [[A]](p0), %bb.0, [[B]](p0), %bb.1
    ; CHECK: %x0 = COPY %3(p0)
     successors: %bb.2, %bb.3
     liveins: %w2, %x0, %x1
 
     %0(p0) = COPY %x0
     %1(p0) = COPY %x1
-    %2(s1) = COPY %w2
+    %4(s32) = COPY %w2
+    %2(s1) = G_TRUNC %4(s32)
     G_BRCOND %2(s1), %bb.2
     G_BR %bb.3
 
   bb.2:
     successors: %bb.3
 
   bb.3:
     %3(p0) = G_PHI %0(p0), %bb.1, %1(p0), %bb.2

test/CodeGen/AArch64/GlobalISel/localizer-in-O0-pipeline.mir

 registers:
 # CHECK: registers:
 # CHECK-NEXT: - { id: 0, class: fpr, preferred-register: '' }
 # CHECK-NEXT: - { id: 1, class: gpr, preferred-register: '' }
 # CHECK-NEXT: - { id: 2, class: fpr, preferred-register: '' }
 # CHECK-NEXT: - { id: 3, class: fpr, preferred-register: '' }
 # CHECK-NEXT: - { id: 4, class: fpr, preferred-register: '' }
 # CHECK-NEXT: - { id: 5, class: fpr, preferred-register: '' }
+# CHECK-NEXT: - { id: 6, class: gpr, preferred-register: '' }
 # The localizer will create two new values to materialize the constants.
-# OPTNONE-NEXT:  - { id: 6, class: fpr, preferred-register: '' }
 # OPTNONE-NEXT:  - { id: 7, class: fpr, preferred-register: '' }
+# OPTNONE-NEXT:  - { id: 8, class: fpr, preferred-register: '' }
   - { id: 0, class: fpr }
   - { id: 1, class: gpr }
   - { id: 2, class: fpr }
   - { id: 3, class: fpr }
   - { id: 4, class: fpr }
   - { id: 5, class: fpr }
+  - { id: 6, class: gpr }
 
 # First block remains untouched
 # CHECK: body
 # CHECK:   %4(s32) = G_FCONSTANT float 1.000000e+00
 # CHECK:   %5(s32) = G_FCONSTANT float 2.000000e+00
 
 # Second block will get the constant 1.0 when the localizer is enabled.
 # CHECK: bb.1.true:

 # OPTNONE: %2(s32) = PHI [[FONE]](s32), %bb.1.true, [[FTWO]](s32), %bb.2.false
 # OPT: %2(s32) = PHI %4(s32), %bb.1.true, %5(s32), %bb.2.false
 # CHECK-NEXT: G_FADD %0, %2
 body:             |
   bb.0 (%ir-block.0):
     liveins: %s0, %w0
 
     %0(s32) = COPY %s0
-    %1(s1) = COPY %w0
+    %6(s32) = COPY %w0
+    %1(s1) = G_TRUNC %6
     %4(s32) = G_FCONSTANT float 1.000000e+00
     %5(s32) = G_FCONSTANT float 2.000000e+00
     G_BRCOND %1(s1), %bb.1.true
     G_BR %bb.2.false
   
   bb.1.true:
     G_BR %bb.3.end
   

test/CodeGen/AArch64/GlobalISel/reg-bank-128bit.mir

bb.0.entry:
     ; CHECK:       - { id: 0, class: gpr
     ; CHECK:       - { id: 1, class: gpr
     ; CHECK:       - { id: 2, class: gpr
     ; CHECK:       - { id: 3, class: fpr
     %0:_(s64) = COPY %x0
     %1:_(s64) = COPY %x1
     %2:_(p0) = COPY %x2
     %3:_(s128) = G_MERGE_VALUES %0, %1
-    %d0 = COPY %3
+    %4:_(s64) = G_TRUNC %3
+    %d0 = COPY %4
 ...

test/CodeGen/AArch64/GlobalISel/regbankselect-default.mir

 # CHECK:   - { id: 3, class: gpr, preferred-register: '' }
 registers:
   - { id: 0, class: _, preferred-register: '' }
   - { id: 1, class: _, preferred-register: '' }
   - { id: 2, class: _, preferred-register: '' }
   - { id: 3, class: _, preferred-register: '' }
   - { id: 4, class: _, preferred-register: '' }
   - { id: 5, class: _, preferred-register: '' }
+  - { id: 6, class: _, preferred-register: '' }
 body:             |
   bb.0:
     successors: %bb.1, %bb.2
     liveins: %w2, %x0, %x1
 
     %0(p0) = COPY %x0
     %1(p0) = COPY %x1
-    %2(s1) = COPY %w2
+    %6(s32) = COPY %w2
+    %2(s1) = G_TRUNC %6
     G_BRCOND %2(s1), %bb.1
     G_BR %bb.2
 
   bb.1:
     successors: %bb.2
 
 
   bb.2:
     %3(p0) = G_PHI %0(p0), %bb.0, %1(p0), %bb.1
     %x0 = COPY %3(p0)
     RET_ReallyLR implicit %x0
 
 ...

test/CodeGen/AArch64/GlobalISel/select-br.mir

 ---
 # CHECK-LABEL: name: conditional_br
 name:            conditional_br
 legalized:       true
 regBankSelected: true
 
 registers:
   - { id: 0, class: gpr }
+  - { id: 1, class: gpr }
 
 # CHECK:  body:
 # CHECK:   bb.0:
 # CHECK:    TBNZW %0, 0, %bb.1
 # CHECK:    B %bb.0
 body:             |
   bb.0:
     successors: %bb.0, %bb.1
-    %0(s1) = COPY %w0
+    %1(s32) = COPY %w0
+    %0(s1) = G_TRUNC %1
     G_BRCOND %0(s1), %bb.1
     G_BR %bb.0
 
   bb.1:
 ...
 
 ---
 # CHECK-LABEL: name: indirect_br

test/CodeGen/AArch64/GlobalISel/select-fma.mir

 body:             |
   bb.0:
     liveins: %w0, %w1, %w2
 
     %0(s32) = COPY %w0
     %1(s32) = COPY %w1
     %2(s32) = COPY %w2
     %3(s32) = G_FMA %0, %1, %2
-    %x0 = COPY %3
+    %w0 = COPY %3
 ...
 

test/CodeGen/AArch64/GlobalISel/select-imm.mir

 
 # CHECK:  body:
 # CHECK:    %0 = MOVi64imm 1234
 body:             |
   bb.0:
     liveins: %w0, %w1
 
     %0(s64) = G_CONSTANT i64 1234
-    %w0 = COPY %0(s64)
+    %x0 = COPY %0(s64)
 ...

test/CodeGen/AArch64/GlobalISel/select-int-ext.mir

 
 ---
 # CHECK-LABEL: name: anyext_s32_from_s8
 name:            anyext_s32_from_s8
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
-# CHECK-NEXT:  - { id: 0, class: gpr32all, preferred-register: '' }
+# CHECK-NEXT:  - { id: 0, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32all, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
-# CHECK:    %0 = COPY %w0
+# CHECK:    %2 = COPY %w0
 # CHECK:    %1 = COPY %0
 body:             |
   bb.0:
     liveins: %w0
 
-    %0(s8) = COPY %w0
+    %2:gpr(s32) = COPY %w0
+    %0(s8) = G_TRUNC %2
     %1(s32) = G_ANYEXT %0
     %w0 = COPY %1(s32)
 ...
 
 ---
 # CHECK-LABEL: name: zext_s64_from_s32
 name:            zext_s64_from_s32
 legalized:       true

 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
-# CHECK:    %0 = COPY %w0
+# CHECK:    %2 = COPY %w0
 # CHECK:    %1 = UBFMWri %0, 0, 15
 body:             |
   bb.0:
     liveins: %w0
 
-    %0(s16) = COPY %w0
+    %2:gpr(s32) = COPY %w0
+    %0(s16) = G_TRUNC %2
     %1(s32) = G_ZEXT %0
     %w0 = COPY %1
 ...
 
 ---
 # CHECK-LABEL: name: zext_s32_from_s8
 name:            zext_s32_from_s8
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
-# CHECK:    %0 = COPY %w0
+# CHECK:    %2 = COPY %w0
 # CHECK:    %1 = UBFMWri %0, 0, 7
 body:             |
   bb.0:
     liveins: %w0
 
-    %0(s8) = COPY %w0
+    %2:gpr(s32) = COPY %w0
+    %0(s8) = G_TRUNC %2
     %1(s32) = G_ZEXT %0
     %w0 = COPY %1(s32)
 ...
 
 ---
 # CHECK-LABEL: name: zext_s16_from_s8
 name:            zext_s16_from_s8
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
-# CHECK:    %0 = COPY %w0
+# CHECK:    %2 = COPY %w0
 # CHECK:    %1 = UBFMWri %0, 0, 7
 body:             |
   bb.0:
     liveins: %w0
 
-    %0(s8) = COPY %w0
+    %2:gpr(s32) = COPY %w0
+    %0(s8) = G_TRUNC %2
     %1(s16) = G_ZEXT %0
-    %w0 = COPY %1(s16)
+    %3:gpr(s32) = G_ANYEXT %1
+    %w0 = COPY %3(s32)
 ...
 
 ---
 # CHECK-LABEL: name: sext_s64_from_s32
 name:            sext_s64_from_s32
 legalized:       true
 regBankSelected: true
 

 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
-# CHECK:    %0 = COPY %w0
+# CHECK:    %2 = COPY %w0
 # CHECK:    %1 = SBFMWri %0, 0, 15
 body:             |
   bb.0:
     liveins: %w0
 
-    %0(s16) = COPY %w0
+    %2:gpr(s32) = COPY %w0
+    %0(s16) = G_TRUNC %2
     %1(s32) = G_SEXT %0
     %w0 = COPY %1
 ...
 
 ---
 # CHECK-LABEL: name: sext_s32_from_s8
 name:            sext_s32_from_s8
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
-# CHECK:    %0 = COPY %w0
+# CHECK:    %2 = COPY %w0
 # CHECK:    %1 = SBFMWri %0, 0, 7
 body:             |
   bb.0:
     liveins: %w0
 
-    %0(s8) = COPY %w0
+    %2:gpr(s32) = COPY %w0
+    %0(s8) = G_TRUNC %2
     %1(s32) = G_SEXT %0
     %w0 = COPY %1(s32)
 ...
 
 ---
 # CHECK-LABEL: name: sext_s16_from_s8
 name:            sext_s16_from_s8
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
-# CHECK:    %0 = COPY %w0
+# CHECK:    %2 = COPY %w0
 # CHECK:    %1 = SBFMWri %0, 0, 7
 body:             |
   bb.0:
     liveins: %w0
 
-    %0(s8) = COPY %w0
+    %2:gpr(s32) = COPY %w0
+    %0(s8) = G_TRUNC %2
     %1(s16) = G_SEXT %0
-    %w0 = COPY %1(s16)
+    %3:gpr(s32) = G_ANYEXT %1
+    %w0 = COPY %3(s32)
 ...

test/CodeGen/AArch64/GlobalISel/select-int-ptr-casts.mir

 # CHECK:  body:
 # CHECK:    %0 = COPY %x0
 # CHECK:    %1 = COPY %0.sub_32
 body:             |
   bb.0:
     liveins: %x0
     %0(p0) = COPY %x0
     %1(s16) = G_PTRTOINT %0
-    %w0 = COPY %1(s16)
+    %2:gpr(s32) = G_ANYEXT %1
+    %w0 = COPY %2(s32)
 ...
 
 ---
 # CHECK-LABEL: name: ptrtoint_s8_p0
 name:            ptrtoint_s8_p0
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr64, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 # CHECK:  body:
 # CHECK:    %0 = COPY %x0
 # CHECK:    %1 = COPY %0.sub_32
 body:             |
   bb.0:
     liveins: %x0
     %0(p0) = COPY %x0
     %1(s8) = G_PTRTOINT %0
-    %w0 = COPY %1(s8)
+    %2:gpr(s32) = G_ANYEXT %1
+    %w0 = COPY %2(s32)
 ...
 
 ---
 # CHECK-LABEL: name: ptrtoint_s1_p0
 name:            ptrtoint_s1_p0
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr64, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 # CHECK:  body:
 # CHECK:    %0 = COPY %x0
 # CHECK:    %1 = COPY %0.sub_32
 body:             |
   bb.0:
     liveins: %x0
     %0(p0) = COPY %x0
     %1(s1) = G_PTRTOINT %0
-    %w0 = COPY %1(s1)
+    %2:gpr(s32) = G_ANYEXT %1
+    %w0 = COPY %2(s32)
 ...

test/CodeGen/AArch64/GlobalISel/select-load.mir

 # CHECK: %0 = COPY %x0
 # CHECK: %1 = LDRHHui %0, 0 :: (load 2 from %ir.addr)
 body:             |
   bb.0:
     liveins: %x0
 
     %0(p0) = COPY %x0
     %1(s16) = G_LOAD  %0 :: (load 2 from %ir.addr)
-    %w0 = COPY %1(s16)
+    %2:gpr(s32) = G_ANYEXT %1
+    %w0 = COPY %2(s32)
 ...
 
 ---
 # CHECK-LABEL: name: load_s8_gpr
 name:            load_s8_gpr
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr64sp, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
 # CHECK: %0 = COPY %x0
 # CHECK: %1 = LDRBBui %0, 0 :: (load 1 from %ir.addr)
 body:             |
   bb.0:
     liveins: %x0
 
     %0(p0) = COPY %x0
     %1(s8) = G_LOAD  %0 :: (load 1 from %ir.addr)
-    %w0 = COPY %1(s8)
+    %2:gpr(s32) = G_ANYEXT %1
+    %w0 = COPY %2(s32)
 ...
 
 ---
 # CHECK-LABEL: name: load_fi_s64_gpr
 name:            load_fi_s64_gpr
 legalized:       true
 regBankSelected: true
 
registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
   - { id: 2, class: gpr }
   - { id: 3, class: gpr }
 
 # CHECK:  body:
 # CHECK: %0 = COPY %x0
 # CHECK: %3 = LDRHHui %0, 32 :: (load 2 from %ir.addr)
-# CHECK: %w0 = COPY %3
+# CHECK: %w0 = COPY %4
 body:             |
   bb.0:
     liveins: %x0
 
     %0(p0) = COPY %x0
     %1(s64) = G_CONSTANT i64 64
     %2(p0) = G_GEP %0, %1
     %3(s16) = G_LOAD %2 :: (load 2 from %ir.addr)
-    %w0 = COPY %3
+    %4:gpr(s32) = G_ANYEXT %3
+    %w0 = COPY %4(s32)
 ...
 
 ---
 # CHECK-LABEL: name: load_gep_1_s8_gpr
 name:            load_gep_1_s8_gpr
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr64sp, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr, preferred-register: '' }
 # CHECK-NEXT:  - { id: 2, class: gpr, preferred-register: '' }
 # CHECK-NEXT:  - { id: 3, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
   - { id: 2, class: gpr }
   - { id: 3, class: gpr }
 
 # CHECK:  body:
 # CHECK: %0 = COPY %x0
 # CHECK: %3 = LDRBBui %0, 1 :: (load 1 from %ir.addr)
-# CHECK: %w0 = COPY %3
+# CHECK: %w0 = COPY %4
 body:             |
   bb.0:
     liveins: %x0
 
     %0(p0) = COPY %x0
     %1(s64) = G_CONSTANT i64 1
     %2(p0) = G_GEP %0, %1
     %3(s8) = G_LOAD %2 :: (load 1 from %ir.addr)
-    %w0 = COPY %3
+    %4:gpr(s32) = G_ANYEXT %3
+    %w0 = COPY %4(s32)
 ...
 
 ---
 # CHECK-LABEL: name: load_s64_fpr
 name:            load_s64_fpr
 legalized:       true
 regBankSelected: true
 

test/CodeGen/AArch64/GlobalISel/select-phi.mir

 body:             |
   bb.0:
     successors: %bb.1, %bb.2
     liveins: %w2, %x0, %x1
     ; CHECK-LABEL: name: test_phi_ptr
 
     %0(p0) = COPY %x0
     %1(p0) = COPY %x1
-    %2(s1) = COPY %w2
+    %6:gpr(s32) = COPY %w2
+    %2(s1) = G_TRUNC %6
     G_BRCOND %2(s1), %bb.1
     G_BR %bb.2
 
   bb.1:
     successors: %bb.2
 
 
   bb.2:
     ; CHECK: %{{[0-9]+}} = PHI %{{[0-9]+}}, %bb.0, %{{[0-9]+}}, %bb.1
     %3(p0) = G_PHI %0(p0), %bb.0, %1(p0), %bb.1
     %x0 = COPY %3(p0)
     RET_ReallyLR implicit %x0
 
 ...

test/CodeGen/AArch64/GlobalISel/select-store.mir

 # CHECK-NEXT:  - { id: 0, class: gpr64sp, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
 # CHECK: %0 = COPY %x0
-# CHECK: %1 = COPY %w1
+# CHECK: %2 = COPY %w1
 # CHECK: STRHHui %1, %0, 0 :: (store 2 into %ir.addr)
 body:             |
   bb.0:
     liveins: %x0, %w1
 
     %0(p0) = COPY %x0
-    %1(s16) = COPY %w1
+    %2:gpr(s32) = COPY %w1
+    %1(s16) = G_TRUNC %2
     G_STORE  %1, %0 :: (store 2 into %ir.addr)
 
 ...
 
 ---
 # CHECK-LABEL: name: store_s8_gpr
 name:            store_s8_gpr
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr64sp, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
 # CHECK: %0 = COPY %x0
-# CHECK: %1 = COPY %w1
+# CHECK: %2 = COPY %w1
 # CHECK: STRBBui %1, %0, 0 :: (store 1 into %ir.addr)
 body:             |
   bb.0:
     liveins: %x0, %w1
 
     %0(p0) = COPY %x0
-    %1(s8) = COPY %w1
+    %2:gpr(s32) = COPY %w1
+    %1(s8) = G_TRUNC %2
     G_STORE  %1, %0 :: (store 1 into %ir.addr)
 
 ...
 
 ---
 # CHECK-LABEL: name: store_zero_s64_gpr
 name:            store_zero_s64_gpr
 legalized:       true

 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
   - { id: 2, class: gpr }
   - { id: 3, class: gpr }
 
 # CHECK:  body:
 # CHECK: %0 = COPY %x0
-# CHECK: %1 = COPY %w1
+# CHECK: %4 = COPY %w1
 # CHECK: STRHHui %1, %0, 32 :: (store 2 into %ir.addr)
 body:             |
   bb.0:
     liveins: %x0, %w1
 
     %0(p0) = COPY %x0
-    %1(s16) = COPY %w1
+    %4:gpr(s32) = COPY %w1
+    %1(s16) = G_TRUNC %4
     %2(s64) = G_CONSTANT i64 64
     %3(p0) = G_GEP %0, %2
     G_STORE %1, %3 :: (store 2 into %ir.addr)
 ...
 
 ---
 # CHECK-LABEL: name: store_gep_1_s8_gpr
 name:            store_gep_1_s8_gpr
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr64sp, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 2, class: gpr, preferred-register: '' }
 # CHECK-NEXT:  - { id: 3, class: gpr, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
   - { id: 2, class: gpr }
   - { id: 3, class: gpr }
 
 # CHECK:  body:
 # CHECK: %0 = COPY %x0
-# CHECK: %1 = COPY %w1
+# CHECK: %4 = COPY %w1
 # CHECK: STRBBui %1, %0, 1 :: (store 1 into %ir.addr)
 body:             |
   bb.0:
     liveins: %x0, %w1
 
     %0(p0) = COPY %x0
-    %1(s8) = COPY %w1
+    %4:gpr(s32) = COPY %w1
+    %1(s8) = G_TRUNC %4
     %2(s64) = G_CONSTANT i64 1
     %3(p0) = G_GEP %0, %2
     G_STORE %1, %3 :: (store 1 into %ir.addr)
 ...
 
 ---
 # CHECK-LABEL: name: store_s64_fpr
 name:            store_s64_fpr

test/CodeGen/AArch64/GlobalISel/select-trunc.mir

 # CHECK:  body:
 # CHECK:    %1 = COPY %0.sub_32
 body:             |
   bb.0:
     liveins: %x0
 
     %0(s64) = COPY %x0
     %1(s8) = G_TRUNC %0
-    %w0 = COPY %1(s8)
+    %2:gpr(s32) = G_ANYEXT %1
+    %w0 = COPY %2(s32)
 ...
 
 ---
 # CHECK-LABEL: name: trunc_s1_s32
 name:            trunc_s1_s32
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: gpr }
   - { id: 1, class: gpr }
 
 # CHECK:  body:
 # CHECK:    %1 = COPY %0
 body:             |
   bb.0:
     liveins: %w0
 
     %0(s32) = COPY %w0
     %1(s1) = G_TRUNC %0
-    %w0 = COPY %1(s1)
+    %2:gpr(s32) = G_ANYEXT %1
+    %w0 = COPY %2(s32)
 ...

test/CodeGen/AArch64/GlobalISel/select.mir

registers:
   - { id: 1, class: gpr }
   - { id: 2, class: gpr }
   - { id: 3, class: gpr }
   - { id: 4, class: gpr }
   - { id: 5, class: gpr }
   - { id: 6, class: gpr }
   - { id: 7, class: gpr }
   - { id: 8, class: gpr }
+  - { id: 9, class: gpr }
+  - { id: 10, class: gpr }
+  - { id: 11, class: gpr }
 
 # CHECK:  body:
 # CHECK:    %wzr = SUBSWrr %0, %0, implicit-def %nzcv
 # CHECK:    %1 = CSINCWr %wzr, %wzr, 1, implicit %nzcv
 
 # CHECK:    %xzr = SUBSXrr %2, %2, implicit-def %nzcv
 # CHECK:    %3 = CSINCWr %wzr, %wzr, 3, implicit %nzcv
 
 # CHECK:    %xzr = SUBSXrr %4, %4, implicit-def %nzcv
 # CHECK:    %5 = CSINCWr %wzr, %wzr, 0, implicit %nzcv
 
 body:             |
   bb.0:
     liveins: %w0, %x0
 
     %0(s32) = COPY %w0
     %1(s32) = G_ICMP intpred(eq), %0, %0
     %6(s1) = G_TRUNC %1(s32)
-    %w0 = COPY %6(s1)
+    %9(s32) = G_ANYEXT %6
+    %w0 = COPY %9(s32)
 
     %2(s64) = COPY %x0
     %3(s32) = G_ICMP intpred(uge), %2, %2
     %7(s1) = G_TRUNC %3(s32)
-    %w0 = COPY %7(s1)
+    %10(s32) = G_ANYEXT %7
+    %w0 = COPY %10(s32)
 
     %4(p0) = COPY %x0
     %5(s32) = G_ICMP intpred(ne), %4, %4
     %8(s1) = G_TRUNC %5(s32)
-    %w0 = COPY %8(s1)
+    %11(s32) = G_ANYEXT %8
+    %w0 = COPY %11(s32)
 ...
 
 ---
 # CHECK-LABEL: name: fcmp
 name:            fcmp
 legalized:       true
 regBankSelected: true
 
 # CHECK:      registers:
 # CHECK-NEXT:  - { id: 0, class: fpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 1, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 2, class: fpr64, preferred-register: '' }
 # CHECK-NEXT:  - { id: 3, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 4, class: gpr32, preferred-register: '' }
 # CHECK-NEXT:  - { id: 5, class: gpr32, preferred-register: '' }
 registers:
   - { id: 0, class: fpr }
   - { id: 1, class: gpr }
   - { id: 2, class: fpr }
   - { id: 3, class: gpr }
   - { id: 4, class: gpr }
   - { id: 5, class: gpr }
+  - { id: 6, class: gpr }
+  - { id: 7, class: gpr }
 
 # CHECK:  body:
 # CHECK:    FCMPSrr %0, %0, implicit-def %nzcv
 # CHECK:    [[TST_MI:%[0-9]+]] = CSINCWr %wzr, %wzr, 5, implicit %nzcv
 # CHECK:    [[TST_GT:%[0-9]+]] = CSINCWr %wzr, %wzr, 13, implicit %nzcv
 # CHECK:    %1 = ORRWrr [[TST_MI]], [[TST_GT]]
 
 # CHECK:    FCMPDrr %2, %2, implicit-def %nzcv
 # CHECK:    %3 = CSINCWr %wzr, %wzr, 4, implicit %nzcv
 
 body:             |
   bb.0:
     liveins: %w0, %x0
 
     %0(s32) = COPY %s0
     %1(s32) = G_FCMP floatpred(one), %0, %0
     %4(s1) = G_TRUNC %1(s32)
-    %w0 = COPY %4(s1)
+    %6(s32) = G_ANYEXT %4
+    %w0 = COPY %6(s32)
 
     %2(s64) = COPY %d0
     %3(s32) = G_FCMP floatpred(uge), %2, %2
     %5(s1) = G_TRUNC %3(s32)
-    %w0 = COPY %5(s1)
+    %7(s32) = G_ANYEXT %5
+    %w0 = COPY %7(s32)
 
 ...
 
 ---
 # CHECK-LABEL: name: phi
 name:            phi
 legalized:       true
 regBankSelected: true

 # CHECK:    bb.1:
 # CHECK:      %2 = PHI %0, %bb.0, %2, %bb.1
 
 body:             |
   bb.0:
     liveins: %s0, %w0
     successors: %bb.1
     %0(s32) = COPY %s0
-    %1(s1) = COPY %w0
+    %3:gpr(s32) = COPY %w0
+    %1(s1) = G_TRUNC %3
 
   bb.1:
     successors: %bb.1, %bb.2
     %2(s32) = PHI %0, %bb.0, %2, %bb.1
     G_BRCOND %1, %bb.1
 
   bb.2:
     %s0 = COPY %2

 # CHECK:      %3 = CSELWr %1, %2, 1, implicit %nzcv
 # CHECK:      %wzr = ANDSWri %0, 0, implicit-def %nzcv
 # CHECK:      %6 = CSELXr %4, %5, 1, implicit %nzcv
 # CHECK:      %wzr = ANDSWri %0, 0, implicit-def %nzcv
 # CHECK:      %9 = CSELXr %7, %8, 1, implicit %nzcv
 body:             |
   bb.0:
     liveins: %w0, %w1, %w2
-    %0(s1) = COPY %w0
+    %10:gpr(s32) = COPY %w0
+    %0(s1) = G_TRUNC %10
 
     %1(s32) = COPY %w1
     %2(s32) = COPY %w2
     %3(s32) = G_SELECT %0, %1, %2
     %w0 = COPY %3(s32)
 
     %4(s64) = COPY %x0
     %5(s64) = COPY %x1
     %6(s64) = G_SELECT %0, %4, %5
     %x0 = COPY %6(s64)
 
     %7(p0) = COPY %x0
     %8(p0) = COPY %x1
     %9(p0) = G_SELECT %0, %7, %8
     %x0 = COPY %9(p0)
 ...

test/Verifier/test_copy.mir

+#RUN: not llc -o - -global-isel -run-pass=none -verify-machineinstrs %s 2>&1 | FileCheck %s
+# REQUIRES: global-isel, aarch64-registered-target
+--- |
+  ; ModuleID = 'test.ll'
+  source_filename = "test.ll"
+  target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
+  target triple = "aarch64-unknown-unknown"
+  
+  define i32 @test_copy(i32 %argc) {
+    ret i32 0
+  }
+
+...
+---
+name:            test_copy
+legalized:       true
+regBankSelected: false
+selected:        false
+tracksRegLiveness: true
+registers:       
+  - { id: 0, class: _, preferred-register: '' }
+liveins:         
+body:             |
+  bb.0:
+    liveins: %w0
+    ; This test is used to catch verifier errors with copys having mismatching sizes
+    ; CHECK: Bad machine code: Copy Instruction is illegal with mismatching sizes
+  
+    %0(s8) = COPY %w0
+...