Add a verifier test to check the access on both sides of COPY are the same

[aditya_nandakumar]

Summary

Only checks if there are no sub registers involved for now.
There are lots of failures currently which need to be fixed by various backends.

Splitting this off from https://reviews.llvm.org/D37640

[igorb]

Failed to build as a shared library -DBUILD_SHARED_LIBS=1

[igorb]

lib/CodeGen/MachineVerifier.cpp

981

For X86 back-end this is not always correct.
I think in case a few register classes with different size are mapped to the same physical registers, it is impossible to identify when the COPY is illegal without some additional target specific info.

def FR32 : RegisterClass<"X86", [f32], 32, (sequence "XMM%u", 0, 15)>;
def VR128 : RegisterClass<"X86", [v4f32, v2f64, v16i8, v8i16, v4i32, v2i64], 128, (add FR32)>;

For example the test fail DAG/GISEL (with the patch)
./bin/llc -O0 -mtriple=x86_64-linux-gnu -verify-machineinstrs -stop-before=legalizer
./bin/llc -O0 -mtriple=x86_64-linux-gnu -global-isel -verify-machineinstrs -stop-before=legalizer

define float @test_add_float(float %arg1, float %arg2) {
  %ret = fadd float %arg1, %arg2
  ret float %ret
}

for the correct MIR. (xmm size is 128)

%0(s32) = COPY %xmm0
%1(s32) = COPY %xmm1
%2(s32) = G_FADD %0, %1
%xmm0 = COPY %2(s32)

Thanks

[qcolombet]

lib/CodeGen/MachineVerifier.cpp

981

I'd argue that when generic types are on one side of a copy, i.e., this is a pre-isel copy, we want the size to exactly match.
Thus, if the target wants to do truncate and whatnot at this point, it needs to use G_TRUNC, G_ANYEXT, and so on.

That being said, the check is too broad. We need to check that only for pre-isel copy. Regular copies require indeed target knowledge (e.g., eflags = copy gpr is valid on x86).

[aditya_nandakumar]

lib/CodeGen/MachineVerifier.cpp

981

I can update the check to do it only when there are generic types involved (ie one of src and dst has a generic virtual register).

[qcolombet]

lib/CodeGen/MachineVerifier.cpp

981

That would be perfect.

[aditya_nandakumar]

Updated to check only if one of Src/Dst has generic virtual registers.

[aditya_nandakumar]

Updated to refactor getRegSizeInBits into TargetRegisterInfo. There are 57 failures in total.

[qcolombet]

LGTM with one suggestion

lib/CodeGen/MachineVerifier.cpp

985

Seeing this pattern being actually the implementation of RegisterBankInfo::getSizeInBits, I would suggest to move the whole RegisterBankInfo::getSizeInBits directly in TargetRegisterInfo.

[aditya_nandakumar]

Also, there are various tests that need to be updated by the various backends, and I can only land once those are updated. I haven't had time to update all the backends.

lib/CodeGen/MachineVerifier.cpp

985

I thought TargetRegisterInfo doesn't have LLT headers pulled in (but looks like it is getting transitively pulled in). I can pretty much move the implementation of RegisterBankInfo::getSizeInBits into TargetRegisterInfo.

[aditya_nandakumar]

moved getSizeInBits to TargetRegisterInfo as suggested by Quentin.

[aditya_nandakumar]

Updated AArch64 tests.

Would it be possible for someone to update the ARM/X86 tests?

[rovka]

Sorry, I just noticed this. I'll have a look at the ARM side of things.

[rovka]

Hi, here's the patch for the ARM GlobalISel tests: https://reviews.llvm.org/differential/diff/119563/
Let me know if there's anything else that needs addressing on ARM.

[aditya_nandakumar]

Thanks Diana.
@igorb would it be possible for you to look at X86 when you get a chance? If not, I can update it.

[zvi]

Hi All,

I will try to cover for Igor in helping with the X86 part of this patch.

AFAIU, the challenge is in getting the call/return lowering working for the case where a function's argument/return-value is a f32 or a f64. Some subtargets will use the FP stack registers (which are 80-bit wide), and other will use XMM's (which are 128-bit wide).
Example:

define float @foo(float %arg1, float %arg2) {
  ret float %arg2
}

Existing state on ToT: llc -mtriple=x86_64-linux-gnu -mattr=+sse2 -global-isel -stop-after=irtranslator

body:             |
  bb.1 (%ir-block.0):
    liveins: %xmm0, %xmm1

    %0:_(s32) = COPY %xmm0
    %1:_(s32) = COPY %xmm1
    %xmm0 = COPY %1(s32)
    RET 0, implicit %xmm0

Here are some options for resolving this issue:

1. Perform the full expansion in X86CallLowering using G_COPY/G_TRUNC and G_ANYEXT/G_COPY. Fix whatever needs fixing to make it work.
2. X86CallLoweing will continue with generation of G_COPY's with incompatible src/dest bit-sizes, but will assign register classes to the virtual registers. This will technically work because the verification check is guarded by the condition 'SrcTy.isValid() || DstTy.isValid()', which will not be met if final register classes are assigned.
3. Introduce an X86-specific COPY instruction which will be pattern-matched in Instruction-Selection. This instruction will be generated instead of G_COPY's.

I haven't tried implementing any of the above options, and given my limited knowledge in GlobalISel, would appreciate any feedback on this.

Thanks, Zvi

[aditya_nandakumar]

I would definitely prefer the sizes to be right coming out of the CallLowering - so option #1.

[aditya_nandakumar]

ping?

[igorb]

Hi Aditya ,
Unfortunately, I don't have time to work on this task

Regards,
Igor

[aivchenk]

Hi Aditya,
I'm going to take this over from Igor; Will do that in Jan

[rtereshin]

lib/CodeGen/MachineVerifier.cpp

984

What if neither of the registers has a class? That would mean, both registers have LLTs. Then, the check won't complain if those LLTs have the same size in bits, but don't exactly match, like v2s32 and v4s16, for instance.

Shouldn't LLTs match exactly if both are valid?

[aditya_nandakumar]

lib/CodeGen/MachineVerifier.cpp

984

Good catch. I should be checking LLTs are equal here. I will update the patch.

[aivchenk]

This comment has been deleted.

lib/CodeGen/MachineVerifier.cpp

976

Here we rely on having operands in MI, while previously in MachineVerifier::visitMachineInstrBefore we could discover that there are not enough operands given. Should we guard that with foundErrors?

test/CodeGen/MIR/X86/machine-verifier.mir
fails due to that reason

[qcolombet]

lib/CodeGen/MachineVerifier.cpp

976

Sounds sensible to me.

lib/CodeGen/TargetRegisterInfo.cpp

455

This function is not consistent with RegisterBankInfo::getSizeInBits in the priority it gives to the different size when several sources are available (RC, Type).

RBI goes Type if any then RC, this one does the opposite and I believe this is not what we want.

At the very least, RBI should use this implementation instead of duplicating the logic.

[aivchenk]

X86 currently had/has three kinds of fails after this patch:

1. test/CodeGen/MIR/X86/machine-verifier.mir
   • should be fixed in this patch
2. Proper G_COPY/G_TRUNC and G_ANYEXT/G_COPY for floating point arguments
   • D42287 fixes that. However, it relies on getRegSizeInBits from this patch. We can either decide to split this patch or to submit it first and D42287 as the next one. The second option is not really good, though
3. The rest of the tests just needed small tweaks or reautogeneration. This is fixed and submitted in rL323209

[aditya_nandakumar]

Updated based on Quentin's/Roman's feedback.
With this patch, only X86 backend fails.

[qcolombet]

LGTM, coordinate with the x86 changes and you should be good to go!

[aditya_nandakumar]

I can push the getSizeInbits change in a NFC change all by itself, and then the verifier change can land separately once X86 finishes it?

lib/CodeGen/TargetRegisterInfo.cpp

455

Good catch. Will fix this.

[aditya_nandakumar]

Pushed the NFC change in r324125.

[aivchenk]

X86 part is submitted. All ready to go :)

include/llvm/CodeGen/TargetRegisterInfo.h

virtual unsigned getRegPressureSetScore(const MachineFunction &MF,
                                           unsigned PSetID) const {
     return PSetID;
   }
 
   /// Get the weight in units of pressure for this register class.
   virtual const RegClassWeight &getRegClassWeight(
     const TargetRegisterClass *RC) const = 0;
 
+  /// Returns size in bits of a phys/virtual/generic register.
+  unsigned getRegSizeInBits(unsigned Reg, const MachineRegisterInfo &MRI) const;
+
   /// Get the weight in units of pressure for this register unit.
   virtual unsigned getRegUnitWeight(unsigned RegUnit) const = 0;
 
   /// Get the number of dimensions of register pressure.
   virtual unsigned getNumRegPressureSets() const = 0;
 
   /// Get the name of this register unit pressure set.
   virtual const char *getRegPressureSetName(unsigned Idx) const = 0;

lib/CodeGen/GlobalISel/RegisterBankInfo.cpp

for (unsigned OpIdx = 0,
     }
     DEBUG(dbgs() << '\n');
   }
 }
 
 unsigned RegisterBankInfo::getSizeInBits(unsigned Reg,
                                          const MachineRegisterInfo &MRI,
                                          const TargetRegisterInfo &TRI) const {
-  const TargetRegisterClass *RC = nullptr;
   if (TargetRegisterInfo::isPhysicalRegister(Reg)) {
     // The size is not directly available for physical registers.
     // Instead, we need to access a register class that contains Reg and
     // get the size of that register class.
-    RC = &getMinimalPhysRegClass(Reg, TRI);
-  } else {
-    LLT Ty = MRI.getType(Reg);
-    unsigned RegSize = Ty.isValid() ? Ty.getSizeInBits() : 0;
-    // If Reg is not a generic register, query the register class to
-    // get its size.
-    if (RegSize)
-      return RegSize;
-    // Since Reg is not a generic register, it must have a register class.
-    RC = MRI.getRegClass(Reg);
-  }
-  assert(RC && "Unable to deduce the register class");
+    // Because this is expensive, we'll cache the register class by calling
+    auto *RC = &getMinimalPhysRegClass(Reg, TRI);
+    assert(RC && "Expecting Register class");
     return TRI.getRegSizeInBits(*RC);
   }
+  return TRI.getRegSizeInBits(Reg, MRI);
+}
 
 //------------------------------------------------------------------------------
 // Helper classes implementation.
 //------------------------------------------------------------------------------
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
 LLVM_DUMP_METHOD void RegisterBankInfo::PartialMapping::dump() const {
   print(dbgs());
   dbgs() << '\n';

lib/CodeGen/MachineVerifier.cpp

if (!DstTy.isValid() ||
                          return false;
                        return true;
                      }))
       report("Generic Instruction G_PHI has operands with incompatible/missing "
              "types",
              MI);
     break;
   }
+  case TargetOpcode::COPY: {
+    if (foundErrors)
+      break;
+    const MachineOperand &DstOp = MI->getOperand(0);
+    const MachineOperand &SrcOp = MI->getOperand(1);
+    LLT DstTy = MRI->getType(DstOp.getReg());
+    LLT SrcTy = MRI->getType(SrcOp.getReg());
+    if (SrcTy.isValid() && DstTy.isValid()) {
+      // If both types are valid, check that the types are the same.
+      if (SrcTy != DstTy) {
+        report("Copy Instruction is illegal with mismatching types", MI);
+        errs() << "Def = " << DstTy << ", Src = " << SrcTy << "\n";
+      }
+    }
+    if (SrcTy.isValid() || DstTy.isValid()) {
+      // If one of them have valid types, let's just check they have the same
+      // size.
+      unsigned SrcSize = TRI->getRegSizeInBits(SrcOp.getReg(), *MRI);
+      unsigned DstSize = TRI->getRegSizeInBits(DstOp.getReg(), *MRI);
+      assert(SrcSize && "Expecting size here");
+      assert(DstSize && "Expecting size here");
+      if (SrcSize != DstSize)
+        if (!DstOp.getSubReg() && !SrcOp.getSubReg()) {
+          report("Copy Instruction is illegal with mismatching sizes", MI);
+          errs() << "Def Size = " << DstSize << ", Src Size = " << SrcSize
+                 << "\n";
+        }
+    }
+    break;
+  }
   case TargetOpcode::STATEPOINT:
     if (!MI->getOperand(StatepointOpers::IDPos).isImm() ||
         !MI->getOperand(StatepointOpers::NBytesPos).isImm() ||
         !MI->getOperand(StatepointOpers::NCallArgsPos).isImm())
       report("meta operands to STATEPOINT not constant!", MI);
     break;
 
     auto VerifyStackMapConstant = [&](unsigned Offset) {

lib/CodeGen/TargetRegisterInfo.cpp

bool TargetRegisterInfo::regmaskSubsetEqual(const uint32_t *mask0,
                                             const uint32_t *mask1) const {
   unsigned N = (getNumRegs()+31) / 32;
   for (unsigned I = 0; I < N; ++I)
     if ((mask0[I] & mask1[I]) != mask0[I])
       return false;
   return true;
 }
 
+unsigned TargetRegisterInfo::getRegSizeInBits(unsigned Reg,
+                                         const MachineRegisterInfo &MRI) const {
+  const TargetRegisterClass *RC{};
+  if (isPhysicalRegister(Reg)) {
+    // The size is not directly available for physical registers.
+    // Instead, we need to access a register class that contains Reg and
+    // get the size of that register class.
+    RC = getMinimalPhysRegClass(Reg);
+  } else {
+    LLT Ty = MRI.getType(Reg);
+    unsigned RegSize = Ty.isValid() ? Ty.getSizeInBits() : 0;
+    // If Reg is not a generic register, query the register class to
+    // get its size.
+    if (RegSize)
+      return RegSize;
+    // Since Reg is not a generic register, it must have a register class.
+    RC = MRI.getRegClass(Reg);
+  }
+  assert(RC && "Unable to deduce the register class");
+  return getRegSizeInBits(*RC);
+}
+
 #if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
 LLVM_DUMP_METHOD
 void TargetRegisterInfo::dumpReg(unsigned Reg, unsigned SubRegIndex,
                                  const TargetRegisterInfo *TRI) {
   dbgs() << printReg(Reg, TRI, SubRegIndex) << "\n";
 }
 #endif

test/CodeGen/AArch64/GlobalISel/legalize-combines.mir

bb.0:
     ; CHECK: [[COPY:%[0-9]+]]:_(s64) = COPY $x0
     ; CHECK: [[COPY1:%[0-9]+]]:_(s64) = COPY [[COPY]](s64)
     ; CHECK: [[ADD:%[0-9]+]]:_(s64) = G_ADD [[COPY1]], [[COPY1]]
     %0:_(s64) = COPY $x0
 
     %1:_(s128) = G_MERGE_VALUES %0, %0
     %2:_(s64) = G_EXTRACT %1, 0
     %3:_(s64) = G_ADD %2, %2
-    $w0 = COPY %3
+    $x0 = COPY %3
 ...
 
 ---
 name:            test_combines_5
 body: |
   bb.0:
     liveins: $w0
 

test/CodeGen/AArch64/GlobalISel/legalize-fptoi.mir

 ---
 name:            test_fptosi_s1_s32
 body: |
   bb.0:
     liveins: $w0
     ; CHECK-LABEL: name: test_fptosi_s1_s32
     ; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY $w0
     ; CHECK: [[FPTOSI:%[0-9]+]]:_(s32) = G_FPTOSI [[COPY]](s32)
-    ; CHECK: [[TRUNC:%[0-9]+]]:_(s1) = G_TRUNC [[FPTOSI]](s32)
-    ; CHECK: $x0 = COPY [[TRUNC]](s1)
+    ; CHECK: [[ANYEXT:%[0-9]+]]:_(s64) = G_ANYEXT [[FPTOSI]](s32)
+    ; CHECK: $x0 = COPY [[ANYEXT]](s64)
     %0:_(s32) = COPY $w0
     %1:_(s1) = G_FPTOSI %0
-    $x0 = COPY %1
+    %2:_(s64) = G_ANYEXT %1
+    $x0 = COPY %2
 ...
 
 ---
 name:            test_fptoui_s1_s32
 body: |
   bb.0:
     liveins: $w0
     ; CHECK-LABEL: name: test_fptoui_s1_s32

test/CodeGen/AArch64/GlobalISel/legalize-itofp.mir

 body: |
   bb.0:
     liveins: $w0
     ; CHECK-LABEL: name: test_sitofp_s64_s32
     ; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY $w0
     ; CHECK: [[SITOFP:%[0-9]+]]:_(s64) = G_SITOFP [[COPY]](s32)
     %0:_(s32) = COPY $w0
     %1:_(s64) = G_SITOFP %0
-    $w0 = COPY %1
+    $x0 = COPY %1
 ...
 
 ---
 name:            test_uitofp_s64_s32
 body: |
   bb.0:
     liveins: $w0
     ; CHECK-LABEL: name: test_uitofp_s64_s32

test/CodeGen/AArch64/GlobalISel/legalize-or.mir

 # NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
 # RUN: llc -O0 -mtriple=aarch64-apple-ios -run-pass=legalizer -global-isel %s -o - | FileCheck %s
 
 ---
 name:            test_scalar_or_small
 body: |
   bb.0:
     liveins: $x0, $x1, $x2, $x3
     ; CHECK-LABEL: name: test_scalar_or_small
     ; CHECK: [[COPY:%[0-9]+]]:_(s64) = COPY $x0
     ; CHECK: [[COPY1:%[0-9]+]]:_(s64) = COPY $x1
     ; CHECK: [[TRUNC:%[0-9]+]]:_(s32) = G_TRUNC [[COPY]](s64)
     ; CHECK: [[TRUNC1:%[0-9]+]]:_(s32) = G_TRUNC [[COPY1]](s64)
     ; CHECK: [[OR:%[0-9]+]]:_(s32) = G_OR [[TRUNC]], [[TRUNC1]]
-    ; CHECK: [[TRUNC2:%[0-9]+]]:_(s8) = G_TRUNC [[OR]](s32)
-    ; CHECK: $x0 = COPY [[TRUNC2]](s8)
+    ; CHECK: [[ANYEXT:%[0-9]+]]:_(s64) = G_ANYEXT [[OR]](s32)
+    ; CHECK: $x0 = COPY [[ANYEXT]](s64)
     %0:_(s64) = COPY $x0
     %1:_(s64) = COPY $x1
     %2:_(s8) = G_TRUNC %0
     %3:_(s8) = G_TRUNC %1
     %4:_(s8) = G_OR %2, %3
-    $x0 = COPY %4
+    %5:_(s64) = G_ANYEXT %4
+    $x0 = COPY %5
 ...
 
 ---
 name:            test_big_scalar_power_of_2
 body: |
   bb.0:
     liveins: $x0, $x1, $x2, $x3
     ; We have a temporary G_MERGE_VALUES in the legalizer that gets

test/CodeGen/MIR/AArch64/intrinsics.mir

 # RUN: llc -mtriple=aarch64-none-linux-gnu -run-pass none -o - %s | FileCheck %s
 
 --- |
 
   define void @use_intrin() {
     ret void
   }
 
 ...
 ---
 # Completely invalid code, but it checks that intrinsics round-trip properly.
-# CHECK: $x0 = COPY intrinsic(@llvm.returnaddress)
+# CHECK: G_INTRINSIC intrinsic(@llvm.returnaddress)
 name:            use_intrin
 body: |
   bb.0:
-    $x0 = COPY intrinsic(@llvm.returnaddress)
+    %0:_(s64) = G_INTRINSIC intrinsic(@llvm.returnaddress)
     RET_ReallyLR
 ...

test/CodeGen/MIR/AMDGPU/intrinsics.mir

 ---
 # Completely invalid code, but it checks that intrinsics round-trip properly.
 name:            use_intrin
 registers:
   - { id: 0, class: _ }
 body: |
   bb.0:
     ; CHECK-LABEL: name: use_intrin
-    ; CHECK: [[COPY:%[0-9]+]]:_(s64) = COPY intrinsic(@llvm.amdgcn.sbfe)
-    %0(s64) = COPY intrinsic(@llvm.amdgcn.sbfe.i32)
+    ; CHECK: %0:_(s64) = G_INTRINSIC intrinsic(@llvm.amdgcn.sbfe)
+    %0(s64) = G_INTRINSIC intrinsic(@llvm.amdgcn.sbfe.i32)
 ...

test/Verifier/test_copy.mir

+#RUN: not llc -o - -global-isel -run-pass=none -verify-machineinstrs %s 2>&1 | FileCheck %s
+# REQUIRES: global-isel, aarch64-registered-target
+--- |
+  ; ModuleID = 'test.ll'
+  source_filename = "test.ll"
+  target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
+  target triple = "aarch64-unknown-unknown"
+  
+  define i32 @test_copy(i32 %argc) {
+    ret i32 0
+  }
+  define i32 @test_copy_type_mismatch(i32 %argc) {
+    ret i32 0
+  }
+
+...
+---
+name:            test_copy
+legalized:       true
+regBankSelected: false
+selected:        false
+tracksRegLiveness: true
+registers:       
+  - { id: 0, class: _, preferred-register: '' }
+liveins:         
+body:             |
+  bb.0:
+    liveins: $w0
+    ; This test is used to catch verifier errors with copys having mismatching sizes
+    ; CHECK: Bad machine code: Copy Instruction is illegal with mismatching sizes
+
+    %0(s8) = COPY $w0
+...

test/Verifier/test_copy_mismatch_types.mir

+#RUN: not llc -o - -global-isel -run-pass=none -verify-machineinstrs %s 2>&1 | FileCheck %s
+# REQUIRES: global-isel, aarch64-registered-target
+--- |
+  ; ModuleID = 'test.ll'
+  source_filename = "test.ll"
+  target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
+  target triple = "aarch64-unknown-unknown"
+  
+  define i32 @test_copy(i32 %argc) {
+    ret i32 0
+  }
+
+...
+---
+name:            test_copy
+legalized:       true
+regBankSelected: false
+selected:        false
+tracksRegLiveness: true
+registers:       
+  - { id: 0, class: _, preferred-register: '' }
+liveins:         
+body:             |
+  bb.0:
+    liveins: $w0
+    ; This test is used to catch verifier errors with copys having mismatching sizes
+    ; CHECK: Bad machine code: Copy Instruction is illegal with mismatching types
+
+    %0(s32) = COPY $w0
+    %1:_(<2 x s16>) = COPY %0
+...