[AMDGPU] Preserve vcc_lo when shrinking V_CNDMASK

[piotr]

Summary

There is no justification for changing vcc_lo to vcc
when shrinking V_CNDMASK, and such a change could
later confuse live variable analysis.

Make sure the original register is preserved.

[arsenm]

llvm/lib/Target/AMDGPU/SIInstrInfo.cpp

3432

This seems like the wrong place to set this. This implies the registers were mismatched before, and this should be fixed up when the shrunk instruction is created (I guess setting it here isn't that much different)

The way we handle wave32 vs. wave64 is super gross, and we really should have separate instructions.

[piotr]

As suggested, moved the code to buildShrunkInst where the instruction is created.

[arsenm]

llvm/lib/Target/AMDGPU/SIInstrInfo.cpp

3475–3480

We have SIInstrInfo::fixImplicitOperands for this

[piotr]

Switched to SIInstrInfo::fixImplicitOperands, thanks.

llvm/lib/Target/AMDGPU/SIInstrInfo.cpp

 static void copyFlagsToImplicitVCC(MachineInstr &MI,
                                    const MachineOperand &Orig) {
 
   for (MachineOperand &Use : MI.implicit_operands()) {
     if (Use.isUse() &&
         (Use.getReg() == AMDGPU::VCC || Use.getReg() == AMDGPU::VCC_LO)) {
       Use.setIsUndef(Orig.isUndef());
       Use.setIsKill(Orig.isKill());
       return;
     }
   }
 }
 
 MachineInstr *SIInstrInfo::buildShrunkInst(MachineInstr &MI,
                                            unsigned Op32) const {
   MachineBasicBlock *MBB = MI.getParent();;
   MachineInstrBuilder Inst32 =
MachineInstr *SIInstrInfo::buildShrunkInst(MachineInstr &MI,
   const MachineOperand *Src2 = getNamedOperand(MI, AMDGPU::OpName::src2);
 
   if (Src2) {
     int Op32Src2Idx = AMDGPU::getNamedOperandIdx(Op32, AMDGPU::OpName::src2);
     if (Op32Src2Idx != -1) {
       Inst32.add(*Src2);
     } else {
       // In the case of V_CNDMASK_B32_e32, the explicit operand src2 is
-      // replaced with an implicit read of vcc. This was already added
-      // during the initial BuildMI, so find it to preserve the flags.
+      // replaced with an implicit read of vcc or vcc_lo. The implicit read
+      // of vcc was already added during the initial BuildMI, but we
+      // 1) may need to change vcc to vcc_lo to preserve the original register
+      // 2) have to preserve the original flags.
+
+      if (Src2->getReg() == AMDGPU::VCC_LO) {
+        for (auto &Use : Inst32->implicit_operands()) {
+          if (Use.isUse() && Use.getReg() == AMDGPU::VCC)
+            Use.setReg(AMDGPU::VCC_LO);
+        }
+      }
+
       copyFlagsToImplicitVCC(*Inst32, *Src2);
     }
   }
 
   return Inst32;
 }
 
 bool SIInstrInfo::usesConstantBus(const MachineRegisterInfo &MRI,

llvm/test/CodeGen/AMDGPU/shrink-instructions-implicit-vcclo.mir

+# RUN: llc -march=amdgcn -mcpu=gfx1010 -run-pass=si-shrink-instructions --verify-machineinstrs %s -o - | FileCheck %s
+
+# Make sure the implicit vcc_lo of V_CNDMASK is preserved and not promoted to vcc.
+---
+
+name:            shrink_cndmask_implicit_vcc_lo
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $vgpr0, $vgpr1
+    ; CHECK-LABEL: name: shrink_cndmask_implicit_vcc_lo
+    ; CHECK: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
+    ; CHECK: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr0
+    ; CHECK: V_CMP_LT_I32_e32 0, [[COPY]], implicit-def $vcc_lo, implicit $exec
+    ; CHECK: V_CNDMASK_B32_e32 0, [[COPY1]], implicit $vcc_lo, implicit $exec
+    %0:vgpr_32 = COPY $vgpr0
+    %1:vgpr_32 = COPY $vgpr0
+    V_CMP_LT_I32_e32 0, %0:vgpr_32, implicit-def $vcc_lo, implicit $exec, implicit-def $vcc
+    %2:vgpr_32 = V_CNDMASK_B32_e64 0, 0, 0, %1:vgpr_32, $vcc_lo, implicit $exec
+    S_NOP 0
+
+...